I don't want people to use my property without permission
Are people using your property without permission or you're just being selfish?
I don't like the idea that others think of me as a no-go person.
Well, it's your property and you should be the gatekeeper to it all. You should always know when someone tries to take what's yours, because you paid for it and it's not theirs.
There's a lot of my neighbors who just do not respect my property and it's not fine.
I suggest you install some surveillance in your house so that you would know who are these people who are using your property without permission, so that you can report them to the police. It may seem drastic, but it is the thing you can do if they continuously do it even if you have already set boundaries with them.