Do you think I can tell my roommate to back off?
Is there a specific reason you want to tell him to back off from you?
There is, but I'd rather not say.
It is ok to tell someone to stop if they keep bothering you. You should be respectful and if you can't work it out you may need to move to keep the peace.
I feel like we need to hang out a little less. It's getting in the way of other things.
Yes, that would be a good reason to back off. We all need our alone time.