My roommate has an orange that I am going to throw away.
Why are you going to throw his orange away?
I don't feel comfortable with the idea of having an orange in my house.
That's not yours to throw away. He bought it and should be able to keep it in the place too.
I don't see what's so special about the orange.
It's not that the orange is special, it's that it belongs to him. If you have a valid reason for not wanting oranges in your place then you should discuss it with them so they can avoid buying any more.