I'm feeling angry at my brother because he said things that I wasn't willing to.
Why did you react to your brother saying saying things you didn't like?
He was wrong to say what he said. I told him, but he wouldn't apologize.
I understand that you were angry, but calling out your brother over minor issues in the way he expresses himself is petty. Ignore your annoyances and let go of pettiness.
He claims he was just joking.
It was wrong of him to joke in a way that would upset you so much, and you should tell him that. Still, it would be a good idea to forgive him so long as he doesn't do it again. Your bond with your sibling is important and shouldn't be eroded by pettiness.