I'm thinking of not inviting a friend over when I hang out with another friend group
Why don't you want to invite the friend?
I want to enjoy a new group of friends.
Your friend might be hurt if he finds out. But it's fine to have different groups of friends for different things.
Na, they are fine since they are open minded and want to meet new friends.
Well, if that's the case, you should just invite your friend to join. If not, I guess it's okay to separate the types of friends you have. I am sure your friend knows other people that you don't know about, which is pretty normal.