I think I am not going to babysit my brothers kids
It's your brothers kid. Why wouldn't you babysit for your brother?
I'm going to cut him out of my life because he's a douchebag, so I won't do what he wants.
Well, it's not your obligation in the end to babysit. It's your choice in the end because you surely have your own things to do. You should talk to him about what you intend to do, though.
Nah, he deserves getting ghosted.
Well, are you sure you want to do this, because there might be no turning back from this. I am sure your brother will get pissed off. That's why I think giving him a head up will be good, but at the end of the day, whatever you decide, I will support your decision.