I'm thinking of calling the police on the man for doing nothing.
Why would you call the police on him if he's done nothing?
Because I don't want to sit with that stranger for a single minute longer.
As you said, that man wasn't doing anything, so calling the police on him is definitely considered rude. Instead of calling the police, why not just leave the area if you really could not stand him?
I want him to get in trouble with the police though.
The man didn't do anything wrong though. Wasting police time is not good. They could be responding to other important matters.