I'm going to ask my neighbor what's the problem when he goes into my front yard
Does it look like he is doing something wrong?
No, just that it looks like he's doing something wrong.
You have the right to ask him that. He should go on your yard without a reason, it's trespassing.
Alright, I'm taking him down, better get my phone ready to record.
That is not a bad idea. If you want him off of your property then you have the right to tell him that. If he does not listen simply call the police and let them do it.