New annotation to add existing Security Groups to ELBs created by AWS cloudprovider

[redbaron]

What this PR does / why we need it:
When K8S cluster is deployed in existing VPC there might be a need to attach extra SecurityGroups to ELB created by AWS cloudprovider. Example of it can be cases, where such Security Groups are maintained by another team.

Special notes for your reviewer:
For tests to pass depends on #45168 and therefore includes it

Release note:

New 'service.beta.kubernetes.io/aws-load-balancer-extra-security-groups' Service annotation to specify extra Security Groups to be added to ELB created by AWS cloudprovider

[k8s-reviewable]

This change is 

[k8s-ci-robot]

Hi @redbaron. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[redbaron]

/cc @justinsb

[jsafrane]

@k8s-bot ok to test

[jsafrane]

I have zero knowledge about AWS load balancers.

/assign @justinsb
/unassign @jsafrane

[redbaron]

anyone?

[justinsb]

Code looks good (sorry for delay)

But .. can you clarify "Example of it can be cases, where such Security Groups are maintained by another team.". I certainly understand that admin teams often impose requirements like this; I'm just a little unsure why they would want an additional SG attached to the ELB, while also letting us create our own and set ingress that way.

Another way of saying this: I would have a better understanding if we were only using the annotated SG, although that would have problems with the source ranges.

[redbaron]

@justinsb limiting source IPs is exactly out case. Separate team maintains SGs and populates them with up to date source IP addresses, then it is imposed as a requirement to have them attached to any public facing ELBs in AWS account. Autocreated SG is then limited to receive requests only from 127.0.0.1/32 or so, so that it doesn't allow any additional IP addresses to talk to ELB. At the same time, autocreated SG acts as a glue SG to allow incoming connections from ELB to kubernetes worker nodes.

[justinsb]

Have you seen #45500 - that has all ELBs share a single SG. I believe the primary use case there is that there is a 500 SG limit per VPC. I'm wondering if that would work for your use case also though.

[justinsb]

So I like the way this works, but I suspect that #45500 actually meets your use case. I'm going to ask them to refactor to use the same code structure you used here, so that if not we can still get yours in also.

[justinsb]

/lgtm

[redbaron]

@justinsb, #45500 and this are complimentary and work very well together. #45500 defines how auto-assigned SG looks like, this PR adds extra SGs to created ELBs

[redbaron]

Should I be targeting this PR to release-1.7 branch instead of master?

[redbaron]

Can not rebase due git clone being broken on windows right now ( #46958), will rebase once it is solved

[grodrigues3]

Readding the approved label since there is a referenced issue and justin approved prior to freeze

[redbaron]

@justinsb , I think it is all green now, only lgtm is missing

[justinsb]

/lgtm

[justinsb]

/retest

[justinsb]

/approve no-issue

[k8s-github-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb, redbaron

Associated issue requirement bypassed by: justinsb

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

• pkg/cloudprovider/providers/aws/OWNERS [justinsb]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[k8s-ci-robot]

@redbaron: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
Jenkins GCE etcd3 e2e	c5fb77b	link	@k8s-bot gce etcd3 e2e test this

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[spiffxp]

/remove-priority P1
/priority important-soon
(I'm not actually sure this is the right priority, just trying to remove the old priority/PN labels)

[redbaron]

@justinb, can you trigger retest it pls? seems unrelated to my changes

[marun]

/retest

[k8s-github-robot]

Automatic merge from submit-queue