Fully support AWS Load Balancer Controller addon

[peter-svensson]

1. What kops version are you running? The command kops version, will display
this information.

Version 1.20.0

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

v1.20.6

3. What cloud provider are you using?

AWS

4. What commands did you run? What is the simplest way to reproduce this issue?

Create a new kops cluster with the AWS Load Balance Controller enabled.

5. What happened after the commands executed?

Failure to use certificates in ALB

6. What did you expect to happen?

All features on the AWS ALB supported

When enabling the AWS Load Balancer Controller today there are some limitations due to lack of permissions. The IAM role (for kops 1.20) does not have enough permissions on the AWS account to support all the features (annotations).
When enabling the add-on with:

spec:
  awsLoadBalancerController:
    enabled: true

The "default" IAM policy document mentioned in the deployment guide for the ALB contains a lot of permissions that are needed to support for example TLS termination in the ALB.

If I for example enable HTTP->HTTPS redirect with:

alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'

in my ingress I will get errors:

{"level":"error","ts":1620292987.8541718,"logger":"controller","msg":"Reconciler error","controller":"ingress","name":"default-ingress","namespace":"","error":"ingress: default/buildtools-examples: AccessDeniedException: User: arn:aws:sts::292662267961:assumed-role/masters.stc-test2.k8s.local/i-06abbb3bfb1fc5667 is not authorized to perform: acm:ListCertificates\n\tstatus code: 400, request id: 720a5427-bbe0-4203-b500-7ed1dca9106f"}

Had some discussions in the kops-dev slack channel regarding a different missing policy #11393 with @olemarkus so decided to create an issue for this.

When testing I found that I (at least) need the following permissions to use TLS and to modify rules:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "acm:DescribeCertificate",
                "elasticloadbalancing:ModifyRule",
                "elasticloadbalancing:DeleteRule"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:<account>:listener-rule/app/*/*/*/*",
                "arn:aws:elasticloadbalancing:*:<account>:listener-rule/net/*/*/*/*",
                "arn:aws:acm:*:<account>:certificate/*"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "acm:ListCertificates",
            "Resource": "*"
        }
    ]
}

But a better way would probably be to look at the policy document linked above.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[olemarkus]

/remove-lifecycle stale

Sorry this issue slipped passed me.
The trick is how to ensure that not any loadbalancer can use any cert. This can probably be solved by adding a tag to certs that LBC can use.

Will this work for you?

[olemarkus]

/kind feature

[peter-svensson]

/remove-lifecycle stale

Sorry this issue slipped passed me.
The trick is how to ensure that not any loadbalancer can use any cert. This can probably be solved by adding a tag to certs that LBC can use.

Will this work for you?

Worth a try at least 😊