I'm going to leave the room that I'm having my lunch break in when my coworker comes into work early
Are you wishing to be somewhere else?
I don't want to deal with his bullshit today.
It's very rude to avoid your coworker like that. I know you want to slack off, but leaving the room when they arrive is uncalled for.
I'm not slacking off. I just don't want to see him in particular.
I know you don't have to like your co-workers, but avoiding them does no one any good. Try to settle things between you two.