Fix aws.profile rendering in cred-provider template

[stmcginnis]

Issue number:

Closes #2885

Description of changes:

The ecr-credential-provider can use a profile for determining which credential settings to use out of ~/.aws. This wasn't getting rendered in the configuration template because it was inside an {{#each}} block that would change the handlebars context so it would look for settings values relative to the setting being iterated.

This fixes that by explicitly referencing the aws.profile setting from the root, ignoring iteration context.

Another issue was that setting aws.profile after setting the credential provider settings would not trigger re-rendering of the credential provider configuration file. This adds an "affected-services" value for the setting to tell it to restart kubelet, which triggers the rendering of the template.

A migration was needed because of this affected-services metadata change.

Testing done:

Set credential provider data:

# apiclient apply << EOF
> [settings.kubernetes.credential-providers.ecr-credential-provider]
> enabled = true
> # (optional - defaults to "12h")
> cache-duration = "30m"
> image-patterns = [
>   # One or more URL paths to match an image prefix. Supports globbing of subdomains.
>   "*.dkr.ecr.us-east-2.amazonaws.com",
>   "*.dkr.ecr.us-west-2.amazonaws.com"
> ]
> EOF

Verified the config file rendered with the default aws.profile value:

# cat /etc/kubernetes/kubelet/credential-provider-config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: CredentialProviderConfig
providers:
  - name: ecr-credential-provider
    matchImages:
      - "*.dkr.ecr.us-east-2.amazonaws.com"
      - "*.dkr.ecr.us-west-2.amazonaws.com"
    defaultCacheDuration: "30m"
    apiVersion: credentialprovider.kubelet.k8s.io/v1alpha1
    env:
      - name: HOME
        value: /root
      - name: AWS_PROFILE
        value: default

Updated the profile setting:

# apiclient set -j '{"aws": { "profile": "foo"}}'

Verified the config file was re-rendered with the correct information:

# cat /etc/kubernetes/kubelet/credential-provider-config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: CredentialProviderConfig
providers:
  - name: ecr-credential-provider
    matchImages:
      - "*.dkr.ecr.us-east-2.amazonaws.com"
      - "*.dkr.ecr.us-west-2.amazonaws.com"
    defaultCacheDuration: "30m"
    apiVersion: credentialprovider.kubelet.k8s.io/v1alpha1
    env:
      - name: HOME
        value: /root
      - name: AWS_PROFILE
        value: foo

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[stmcginnis]

This should probably be 1.13.0, 1.13.1 in case we want to backport and do a stable release for this. I'll update it to make cherry-picking to the 1.13.x branch easier. Then if we do a point release it is there. If not, easy enough to change back to 1.14.0 on the develop branch.

[webern]

Out of curiosity, why was the @root. addition needed?

[stmcginnis]

Yeah, that was the tricky, key part of this.

Because this is inside an {{#each}} block, that means handlebars iterates through "each" of the settings.kubernetes.credential-providers values in that collection.

What's not immediately obvious is that it also means you are within the context of those iterated values. So that's why on the next line you can do:

{{#if this.enabled}}

instead of some sort of indexing operation into the collection. But that's also where the subtle gotcha is.

It meant that later on with the line:

{{#if settings.aws.profile}}

it was actually looking at settings.kubernetes.credential-providers[x].settings.aws.profile. Since we're inside the each block, we actually need to break out of that local context to get to the real settings.aws.profile.

Handlebars allows you to do this in two different ways - either relationally (../../aws.profile) or by explicitly going from the anchor like what has been done here. This looks a lot cleaner, and probably safer, so I went with the @root route.