I'm going to leave my friend to finish work
How could you leave your friend hanging like that?
There are rumors he is trying to steal my girl, I don't want my girl to be around him.
Leaving your friend based on some rumors is a poor showing, especially as a friend. Instead, you should properly communicate with him and see if these rumors have any substance to them.
Well fine, I'm not going to do that and I'm going to leave.
Just because you heard a rumor, doesn't mean it's true. Sometimes people can be really cruel and just want to ruin relationships, so why not stick by your friend and try to figure it out together?