I'm going to pretend to not speak english when people try and panhandle money from me
Do you do this often?
No, but I need to start to. The number of panhandlers around town has increased.
Lying to someone just to get them to leave you alone is a poor way of communicating that you don't want to deal with them. You should just be up front and honest about not giving panhandlers any money and then go about your business.
But they don't deserve my attention.
You are wrong here. Just show them the tiniest bit of respect and decline giving them money if you want.