My husband and I have a kid and I'm going to tell him that his dad can't babysit.
Is there a reason you believe that your husband's father shouldn't babysit?
He's too old to be able to take care of our kid properly, and it might be dangerous.
It's reasonable that you don't want your partner's dad to babysit your kid if you think he's not capable of handling it. You have to do what's best for your child, and you're allowed to not want certain people babysitting if you think they're not able.
I'm not ageist, but I don't trust his dad with our son. I don't like his dad. I think he's racist and I believe he will harm our son.
It seems that you have valid reasons for not wanting your husband's father to babysit your kid and it's okay for you to hold firm in your position. Your priority as a parent is to make sure your child is safe and taken care of.