Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not modify ELB security group #49445

zihaoyu opened this issue Jul 22, 2017 · 12 comments · Fixed by #62774

Do not modify ELB security group #49445

zihaoyu opened this issue Jul 22, 2017 · 12 comments · Fixed by #62774


Copy link

@zihaoyu zihaoyu commented Jul 22, 2017


/kind feature

What happened:

We run Kuberentes on AWS. We are trying out LoadBalancer service type.

We use Terraform heavily for managing infrastructure on AWS. As a result we have several shared security groups with "perfect" rules. One of them is supposed to be attached to ELBs so our own IPs, partners, etc are whitelisted. Right now, it seems there isn't a way for Kubernetes to use the security group as is. There are several alternatives:

  • Pass my existing security group as annotation. However, Kubernetes still tries to create one first, or use a global security group. If I don't provide any IP range, Kubernetes will whitelist This seems insecure.
  • Provide my own IP ranges as either annotation or service.Spec.LoadBalancerSourceRanges. Then Kubernetes will create a security group for this ELB with the given IPs. This means I need to duplicate my IPs in all microservices.
  • Provide my own IP ranges as either annotation or service.Spec.LoadBalancerSourceRanges, and use a global security group. Then Kubernetes will modify the global SG with the given IPs. This seems problematic when lots of microservices share one SG but some want different IPs whitelisted.

I want to entertain the idea below:

  • Allow users to sepcify another security group to kube-controller-manager or Service object, which Kubernetes will take it and attach it to ELBs as is.
  • If such security group is provided, skip other IP whitelisting steps.

What you expected to happen:

I can give Kubernetes existing security group(s). Kuberentes just attaches the security group(s) to the managed ELBs without modifying the rules.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

It may already be possible to achieve this, and I'm not aware of. Or there are other concerns so that Kubernetes didn't choose to implement this feature.


  • Kubernetes version (use kubectl version): v1.7.1
  • Cloud provider or hardware configuration**: AWS
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:
Copy link

@zihaoyu zihaoyu commented Jul 22, 2017

/sig aws

Copy link

@nbutton23 nbutton23 commented Jul 28, 2017

PR #49805 may help with this. . .

Copy link

@KIVagant KIVagant commented Dec 12, 2017

This problem could be really critical if one has the Kubernetes dashboard installed. If for some reasons the kubernetes-dashboard service recreates AWS ELB, the related security group will be recreated as well with the "" input/output masks and then botnets could find the ELB and attack your cluster.
JFYI, in our case in the one of clusters with K8s 1.5, the deployment and replication controller were installed from the dashboard using this vector of attack. Then special pods were created and root volume mounted inside. The system crontab was modified by processes in these pods and the crypto miner appeared in all K8s nodes on the system level. After this another pods were installed and the miner was executed inside the Kubernetes pods in parallel with the system level.

Copy link

@winjer winjer commented Dec 18, 2017

just had some downtime on a production cluster because of this. i really need it too ;)

Copy link

@yossip yossip commented Mar 15, 2018


Copy link

@KIVagant KIVagant commented Mar 15, 2018

/kind feature

As for me, it isn't a feature, it's a kind of bug.

Copy link

@justinsb justinsb commented Apr 30, 2018

Some comments on #62774 which may be relevant - please comment if an alternative annotation that did replace the default security-groups (as opposed to adding to them) would/wouldn't meet your use-case: #62774 (comment)

Copy link

@johnzheng1975 johnzheng1975 commented May 24, 2018

For me, I don't add any annotation, I just want to change its security group rule.
After it will create a aws elb, and a specificed secruity group with, I change to xx.xx.xx.xx/32.
However, the security group rule will change back to after some minutes.

Is this by design?
Is there any way for me keep my changes, thanks.

Copy link

@Raffo Raffo commented May 24, 2018

I guess this is part of how the controller works. If you look at #62774 , the PR fixes this problem by allowing you to specify the behaviour you want with an annotation. For the way Kubernetes works, the ELB is owned by Kubernetes and you should never be forced to modify the resource manually, thing that ultimately will not work.

Copy link

@fejta-bot fejta-bot commented Aug 22, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

Copy link

@Raffo Raffo commented Aug 22, 2018

/remove-lifecycle stale

This is still relevant cause #62774 is not merged yet.

k8s-ci-robot added a commit that referenced this issue Oct 1, 2018
Fixes #49445 by not adding the default SG when using SG annotation (AWS)
goodluckbot added a commit to goodluckbot/kubernetes that referenced this issue Nov 11, 2018
Copy link

@rehevkor5 rehevkor5 commented May 22, 2019

This is a good change. The situation pre-1.13.0 is somewhat surprising because explicitly setting loadBalancerSourceRanges to an empty array like so:

  loadBalancerSourceRanges: []

has no effect, and the Security Group Ingress is still added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.