I don't want to have any contact with my father's family
Why do you want to stop having contact with your them?
Because my dad died of cancer when I was a child.
If your father had a disease that killed him, it's not his fault that his life ended. You should stay in touch with family members while they're suffering, and make the last days of their lives better so that you don't have such hard feelings for them when they pass.
My father left me at a young age to try and take care of my ill mother. I needed him much more than her.
It's not anyone's fault that he died, and it's not fair to his family to not see you because of it. Think of how they would feel, not being able to see you grow up and see your dad in you? You never know if you'd actually like them all!