I don't see it being a benefit in communities in that I mod (hentai, paizuri, tentaclehentai, and hentaigif), so I'm not personally keen on it but I can see why larger communities might want it. I'm no longer as hopeful that a more active hentai-oriented community will develop here, so though I voted "No", I'm a little apathetic about the choice now, personally.
From what I recall, some users use downvotes for more than just spam, like disliking things. This creates a problem when people browse the Local feed and downvote stuff they don't like even if they aren't even subbed to that community. This impacts niche communities and their posters/creators who would get more downvotes from people seeking more mainstream content even if it would otherwise fit that community.
If the goal is just to ward off spam, then unless reports aren't enough, I'd keep downvotes off.