I'm going to try to get my son's father's rights terminated
Why are you going to try to get your son's father's rights terminated?
Because he doesn't deserve to be his father, he should never have been considered as a father. He proved to me that he was not responsible and will never be responsible.
Parents are expected to look out for their child's best interests. Sometimes this means keeping them away from an irresponsible and possibly even dangerous parent. Although this might upset your child in the short term, it may be best for them in the long term.
I know he has no right to be in my child's life.
If you think he's not fit to be in your child's life, then I think getting his parental rights taken away is a smart idea. Your number one responsibility is to your child, no one else.