I'm going to tell my classmates to back off the racist professor
How do you plan on making them stop?
I will try to be rude to them.
It sounds like your professor might have said some things that were taken as racist. Instead of stopping your classmates, meet with your professor and tell him how his comments were received so that he has a chance to resolve the situation with the class before it gets out of hand.
HE probably is, how do I even handle this then?
Talk directly with him about what other people are saying. You can't please everyone but it is good to make them aware of what is happening.