I'm going to refuse to pick up after my roommate's birthday party
Why would you refuse to pick up after your roommate's birthday party?
It's because he has done the same thing to me before
It's polite to help out with post-party clean up. If you don't, it could come off as immature and having bad manners. People in your social circle will talk about you behind your back and likely not invite you to any more events or parties.
I'm not scared, I'm gonna do it anyways. He's going to get mad at me afterwards.
If you help out with clean up, I'm sure he will appreciate it. Who knows, he might even help you out next time you need help. It's good to work together with your roommates to keep the place you live clean.