Ability to change default master/nodes security groups

[bzuelke]

1. Describe IN DETAIL the feature/behavior/change you would like to see.
So I have an interesting dilemma, We've deployed out clusters utilizing Kops default security groups which leave the egress wide open. For Audits we obviously can't have this in place so I was seeing if there was or start a dialog to be able to not just have additionalSecurityGroups but to have a nodeSecurityGroup and masterSecurityGroup function for the cluster spec to define different SG's when needed without being bound by initial creation. I'm more than happy to give more details if needed.

[DerekHeldtWerle]

Hi @bzuelke, have you checked out #3625?

[bzuelke]

Hey @DerekHeldtWerle, I have seen that PR. Unfortunately the use case I'm looking for is to essentially edit existing already in place default node and master security groups mainly for CIS. As it stands without defining a SG rules for the IG's it leaves the egress completely open. So when I want to limit the amount of outbound traffic over specific ports, even with using additional security groups the completely open egress takes precedence over the in place SG rules we need to abide to. I was thinking about the ability to possibly add variables to the securitygrouprules.go to be able to set ports or something like that to iterate through. Haven't conceptualized it yet.

I've created a work around for the time being for my case where I recompiled my own kops binary with hardcoded egress rules for nodes IG for 443 outbound to 0.0.0.0/0 over TCP. I'm hoping to be able to find time to look into adding in variables to iterate through the nodes/master SG rules.

[ivans3]

hi @bzuelke i am in a similar situation as you, can you share the changes you made to kops src code?
Thanks

[ms4720]

+1

[bzuelke]

@ivans3 Sorry for the lack in response. Was on vacation. I worked this week on a better solution and am still in testing now, but I've managed add in IgSecurityGroup to the instance groups code to define in the IG's Spec for kops. If thats defined it will clear out the existing security groups for the launch configuration and apply the defined IgSecurityGroup and the additionalSecurityGroups if those are defined as well. So it will remove the default nodes SG and apply the new one. I'll post the files that I edited once I feel like it's a solid change. Also I've been using kops 1.9.1 but would be a simple copy paste to other releases.

[ms4720]

@bzuelke any chance of turning this into a pull to get merged?

[bzuelke]

Totally @ms4720 I'll hopefully get this in soon. Testing went well in my use case yesterday.

[bzuelke]

Haven't forgotten about this as I've been held up by other things. Hoping soon I can get this finished up.

[bzuelke]

This is in place in kops 1.11. #5744 I'm going to close this as it's already merged in that PR.

[ivans3]

Hi @bzuelke did this resolve your issue? I gave the new feature a try but found that in order to set outgoing security group rules on the main kops-generated "masters" security group, i would have to take full ownership of it which would be complicated and potentially break adding new InstanceGroups etc.

[ugurarpaci]

This still seems not to resolve the issue. At least not for egress rules. Kops should not touch the egress rules when we override the default security group. Ingress rules are fine.

[bzuelke]

My issue came down to specific instance groups that needed specific egress and ingress rules. Since kops only used the default nodes SG for all instance groups I needed the ability to override the node SG on my other instance groups. The securityGroupOverride did the trick for me in this case. But yes I do not manage masters nor do I manage the default nodes SG's. So in that case might not be the end all be all solution here if you're looking to manage master SG rules.

[ugurarpaci]

I have secured the ingress and egress rules one by one, and pass this security group to kops during provisioning the cluster. Even if I maintain these rules, it is okay. However, it will still be good to mention these rules (all the ingress rules on masters and workers). It will be a good list for air gap deployments or on complicated set of subnets.