I'm going to exclude a non-member of the group
Why would you do that?
As long as you are here, I will not exclude anyone.
I don't think that your actions should depend on whether I can see you or not.  It's okay to have small friend groups. But it's rude to exclude people just because you can.
I'm doing it for the group, for myself and for my friends.
You should see how your friends feel about letting another member into the group.  If they all want the group to be small and close-knit, then that is fine.  But you should not be excluding people unless you have a very good reason.