How about I tell this annoying person to go away?
Why do you want this person to go away?
Because I'm an extrovert and he's an introvert.
If your uncomfortable with this person, you should level with them. Everyone can't like everyone.
I'm going to get him out of my life since it's my problem that I'm annoyed by him.
Life is just like that sometimes and people aren't meant to be friends. Don't be rash with them and simply tell them that you don't feel like things are going smoothly. Maybe introduce them to someone else you know that could be a better fit.