I'm not going to attend a family dinner.
What makes you not want to attend a family dinner?
Because my family is so stupid every time I go to dinner
Putting yourself first is okay, especially for a family event that stresses you out. Be polite about your refusal and don't call them stupid!
I'm glad you understand my point, and I'll be sure to be polite.
It's great that you're able to look out for yourself, and identify the things that make you upset. I'm sure your family will be understanding about the fact that you want to skip this family dinner.