What do you call it when some gay guys do a drive by? A fruit roll-up
Why would you call gay guys fruits?
Where is the joke? It's obvious gay men don't have sex with women. You should be able to see the humour in a  Fruit Roll Up  Drive-By.
Calling a gay person a fruit seems more like a slur to me. I am not sure but i would assume that someone who is homosexual would take offense to being called a fruit.