I feel like slapping a girl without any remorse
You don't mean that do you? Why would you want to slap her?
Eh... I'm not sure. She can be so rude sometimes.
Slapping another person is wrong, as is causing any kind of physical harm and possible injury. If someone is bothering or upsetting you should try to talk to that person and work out the issue, instead. Using violence is never the answer to a social problem.
Is it wrong to want to punch someone?
Yes it is wrong, and wanting to punch someone is even worse than wanting to slap them, since you could cause much more damage this way. I hope that you can talk to people who are being disagreeable and try to get them to improve. But if that doesn't work, there is still no excuse for ever using violence.