We're having a cookout to celebrate my brother coming home, but for the sake of my mother's mental health there are a few family members that I am not going to invite.
Don't you think that your brother would want them there?
He might, but my mom is too emotionally shaken by the truth of his coming home for me to invite them all.
It's good that you are worried about your mother's mental health, but this day should be able your brother. Inviting all of his family members may be what he wants and that's the most important thing to make his coming home easier on him, and your mother will understand that.
Maybe I should just not invite my mother then if that's what you say.
I think you should give your mother an invitation and she can come if she wants, but make sure it's only if she's comfortable. Perhaps you, your brother and your mom can have your own celebration that would be easier for her and she can come to the cookout if she chooses.