It's interesting how people try to redefine the word joke. If there's nothing funny about it, it's not a joke. So maybe they mean that they were trolling just for fun, which is not the same thing.

The second point is that if they say something horrible and then you call them an asshole and then they complain that you've gotten too sensitive, which is a very common chain of events, it actually shows that they've gotten too sensitive. If you're not supposed to care about what they said, why should they care about what you said? ... But of course actually they don't care about what you said, and they're just grandstanding. But even if we accept their internal logic just for the sake of argument, it still makes them giant assholes.