I think there's also a bit of not wanting people you don't like to enjoy the same things you do. It's selfish, sure, but it can be nice when you feel like the fans of a certain thing have compatible personalities. I seem to recall hearing how Kurt Cobain was really upset when the bigoted jerks who made fun of him in his younger days suddenly acted like they loved him once he became famous. Not that I've ever been a particular fan of Nirvana, but I find Cobain to be a sympathetic figure.