I feel like I should insist my brother stay in his room while sick
Why would you insist your sick brother stay in his room?
I'll feel worried every time I hear him cough. His flu could be very contagious.
I understand wanting to avoid sick people, but this is your brother you're talking about. He has just as much a right as you do to use the full house.
I feel bad enough seeing him feeling bad.
I'm glad to hear that you are concerned for your brother's health and you just want to stay healthy too. But don't you want him to recover happily as well? How is he going to feel if you always tell him to stay away?