Fix AWS NLB security group updates

[kellycampbell]

This corrects a problem where valid security group ports were removed
unintentionally when updating a service or when node changes occur.

What this PR does / why we need it:

There is a bug in the existing logic that causes valid security group port mappings to be removed incorrectly.

Which issue(s) this PR fixes:

Fixes #60825

Special notes for your reviewer:

This still needs some unit tests. The existing loadbalancer tests are very minimal. Any pointers to how to mock security group actions?

Release note:

Fix AWS NLB security group updates where valid security group ports were incorrectly removed
when updating a service or when node changes occur.

Root cause was an assumption that the list of security groups was actually a set.

Here's some psuedo code of the cause:

existingGroups = [g1:80, g1:80]  // <-- NOT a Set
desiredGroups = [g1:80]

portChanges = { desiredGroups : ports }
// portChanges = [g1:80 = true]

for g in existingGroups:
    desiredSet.Insert(portChanges[g] == true)
    // iter1: desiredSet = [ g1:80 ]
    // iter2: desiredSet = [ ]
    existingSet = portsForGroup() 
    // i1: existingSet = [ g1:80 ]
    // i2: existingSet = [ g1:80 ]

    alreadyExists = desiredSet.Intersection(existingSet)
    removeAlreadyExistsFromPortChanges()
    // i1: portChanges = []
    // i2: portChanges = []

    notDesired = existingSet.Minus(desiredSet)
    // i1: notDesired = []
    // i2: notDesired = [g1:80]
    setPortChangeToFalseForNotDesired

[kellycampbell]

Really simple test case that reproduced the issue easily for me:

1. Create two services with NLB load balancer - SG's get update to allow the node ports to 0.0.0.0/0
2. Alter one of the services e.g. by adding a new label in the metadata
3. Apply the alteration. The old controller logic removes its node port from the SG.
4. You can continue toggling the SG just by applying label updates to the service.

[kellycampbell]

/sig aws
/kind bug

[rosskukulinski]

/assign @kris-nova

[rosskukulinski]

@kris-nova related to 1.12/1.13 AWS goals: kubernetes/enhancements#423 (comment)

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: helpdesk@rt.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[geojaz]

/lgtm

[geojaz]

/lgtm

[Freyert]

/test pull-kubernetes-e2e-kops-aws
/test pull-kubernetes-e2e-gce-100-performance

[Freyert]

What's the procedure with merging this? I see it needs the lgtm label again, but does someone have to push the button afterward?

[acaire]

Thanks @kellycampbell this is great. We're running private clusters in VPCs with multiple CIDRs and the health check ingress rules only factor in the main CIDR, not the CIDR for the relevant target group instances. I'm still stepping through the code to see where the CIDRs originate from, just wanted to give you a heads up as I was hoping to figure out a fix before this is merged.

[kellycampbell]

/label priority/important-soon

@micahhausler I'd like to get this bugfix into the v1.13 milestone and release if possible.

Also I would like to work on backporting it to 1.12 and 1.11 since NLB doesn't work at all for us without this and we'd like to use one of the features NLB provides.

[kellycampbell]

/assign @d-nishi

[justinsb]

Is there an else block missing here?

OK to do in a follow-on PR

[kellycampbell]

Yeah, that looks like it was overlooked in the original code from @micahhausler f9445b9#diff-b390ae3de185e9f13a631a5c07b8f3ffR646

[justinsb]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb, kellycampbell, micahhausler

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/cloudprovider/providers/aws/OWNERS [justinsb]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[brenix]

Did this get cherry-picked to the 1.11 release? I see the cherry-pick for 1.12, but not 1.11. It would be nice if it was as the latest release kops supports is 1.11

[kellycampbell]

Did this get cherry-picked to the 1.11 release? I see the cherry-pick for 1.12, but not 1.11. It would be nice if it was as the latest release kops supports is 1.11

I don't think this has been picked into 1.13 yet either.

[d-nishi]

This has not been delivered yet.

Slated for 1.14 and is with @M00nF1sh

[M00nF1sh]

Did this get cherry-picked to the 1.11 release? I see the cherry-pick for 1.12, but not 1.11. It would be nice if it was as the latest release kops supports is 1.11

I didn't see any reason that this cannot be cherry-picked into v1.11 and v1.13, i'll create two PRs for them 😄