I'm going to keep my friend's family from contacting him
Why do you want to keep your friend's family from contacting him?
I hate his family because they don't trust me so it's fine if they can't get in touch with him.
You are interfering in other people's business and you need to stay out of family situations. It is not appropriate for you to separate family members or get keep anyone from contacting their own family. You are being selfish and unfair to your friend.
I don't think it's unfair at all.
Look, if you keep trying to control him, he'll no longer want to be your friend. In other words, butt out.