AWS LBC cannot delete Security Groups it creates

[johngmyers]

/kind bug

1. What kops version are you running? The command kops version, will display
this information.

master branch. Broken code appears to be in release-1.23 branch as well

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

1.21.7

3. What cloud provider are you using?

AWS

4. What commands did you run? What is the simplest way to reproduce this issue?

Examine the kops-aws-aws-load-balancer-controller e2e test.

Alternatively:

1. Create a cluster with the AWS Load Balancer controller.
2. Create an Ingress backed by an ALB.
3. Delete the ingress

5. What happened after the commands executed?

The ALB controller is unable to delete the "k8s-traffic-" ingress for the ALB due to lack of IAM permissions. This is because the security group is missing the KubernetesCluster tag.

6. What did you expect to happen?

Security group is tagged with the tags that kOps passed AWS LBC through the default-tags flag. LBC is able to delete the security groups it creates.

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.

9. Anything else do we need to know?

Introduced by #12920.

Filed upstream as kubernetes-sigs/aws-load-balancer-controller#2424

[hakman]

IMO, anything related to LBC should not block anything.
If you want, we can revert the commit in 1.23.

[olemarkus]

We can't ship a broken addon. But we can revert the upgrade. We are still waiting for a few other addons so there is time if upstream moves fast.

[hakman]

We don't want to ship a broken addon, but this is already broken. It doesn't work at all for k8s 1.22+ and we still ship it.
We can also make ec2:DeleteSecurityGroup to unconditional action.

[johngmyers]

Not working at all for k8s 1.22+ is not "broken". It still worked for k8s 1.21 and earlier. With this issue it stopped working for k8s 1.21 and earlier.

I don't think we should open up the RBAC. We could change the condition to use their tag, but the controller would still be failing to tag the SG with the admin-provided tags.

[hakman]

I still see it as "broken". I don't mind reverting the update. I don't mind even removing it from the kOps addons.

[justinsb]

@olemarkus mentioned that the new version of LBC has been merged so can we close this?

[johngmyers]

This can probably be closed, but I can't test it right now.