I'm banning someone from one of my subreddit and he started telling me that I was abusing my mod powers.
Have you ever banned him from this subreddit before?
Yes, once for being a moron.
If it's your subreddit, it's okay to ban people. As long as you clearly state what the rules are and why he violated them, you're well within your power to do so.
I'm going to continue to ban him since he insulted me and called me abusive.
Maybe you should permanently ban him if he is harassing you? You're allowed to use your moderation powers to remove someone who is negative from your community.