models, pluto: generate 'provider-id' for aws-k8s variants

[etungsten]

Issue number:

#3013

Description of changes:

    migrations: add 'aws-k8s-provider-id-gen' migration
    
    This migrates the new 'setting-generator' for 'provider-id' in aws-k8s
    variants.

    models: generate 'provider-id' for aws-k8s variants

    pluto: add 'provider-id' subcommand
    
    Adds a 'provider-id' subcommand to generate the appropriate provider ID
    for the node.

    imdsclient: add 'fetch_zone' helper
    
    Refactored out 'fetch_identity_document' from 'fetch_region' so it can
    used in 'fetch_zone' as well.

Testing done:

Launched 10 aws-k8s-1.26 x86 nodes into subnet with hostname type set to "resource name"
All nodes became ready and none of them have taints:

$ kubectl get nodes -A
NAME                                             STATUS   ROLES    AGE   VERSION
i-0210e782988135b43.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0313c9cacdfdb4191.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-050c48d25af8b59ca.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-05131e8a5c362dbe8.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0705ad44abbb087e4.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-077f02c192511efce.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0aff9f14092c33c3d.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0b69fdec83a27fbc0.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0e45a9dddbbb6a9b2.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822
i-0f8db6a09d54c4081.us-west-2.compute.internal   Ready    <none>   10m   v1.26.2-eks-b106822

$ kubectl get nodes -o json | jq ".items[]|{name:.metadata.name, taints:.spec.taints}"
{
  "name": "i-0705ad44abbb087e4.us-west-2.compute.internal",
  "taints": null
}
{
  "name": "i-0aff9f14092c33c3d.us-west-2.compute.internal",
  "taints": null
}
{
  "name": "i-0b69fdec83a27fbc0.us-west-2.compute.internal",
  "taints": null
}
{
  "name": "i-0e45a9dddbbb6a9b2.us-west-2.compute.internal",
  "taints": null
}
{
  "name": "i-0f8db6a09d54c4081.us-west-2.compute.internal",
  "taints": null
}
...

After terminating 5 instances, and the nodes automatically gets de-registered.

$ kubectl get nodes -A
NAME                                             STATUS   ROLES    AGE   VERSION
i-0705ad44abbb087e4.us-west-2.compute.internal   Ready    <none>   13m   v1.26.2-eks-b106822
i-0aff9f14092c33c3d.us-west-2.compute.internal   Ready    <none>   13m   v1.26.2-eks-b106822
i-0b69fdec83a27fbc0.us-west-2.compute.internal   Ready    <none>   13m   v1.26.2-eks-b106822
i-0e45a9dddbbb6a9b2.us-west-2.compute.internal   Ready    <none>   13m   v1.26.2-eks-b106822
i-0f8db6a09d54c4081.us-west-2.compute.internal   Ready    <none>   13m   v1.26.2-eks-b106822

Tested on aws-k8s-1.25 and saw the same results.
Tested on aws-k8s-1.24 and saw the same results.
Tested on aws-k8s-1.23 and saw the same results.
Tested on aws-k8s-1.22 and saw the same results.

Migration testing:

Launched 1.13.2 aws-k8s-1.26 x86 nodes and upgraded the host to image that includes these changes:

bash-5.1# updog whats                                                                                                                                                                                                                                         
aws-k8s-1.26 1.14.0                                                                                                                                                                                                                                           
bash-5.1# updog update -i 1.14.0 -r -n                                                                                                                                                                                                                        
Starting update to 1.14.0                                                                                                                                                                                                                                     
Cannot schedule shutdown without logind support, proceeding with immediate shutdown.                                                                                                                                                                          
Update applied: aws-k8s-1.26 1.14.0                                                                      

The host boots into 1.14.0 just fine, checked datastore and found the new setting-generator metadata for settings.kubernetes.provider-id:

bash-5.1# cat /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/provider-id.setting-generator
"pluto provider-id"
bash-5.1# cat /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/provider-id
"aws:///us-west-2d/i-033b9d63b15cb5420"
bash-5.1# pluto provider-id
"aws:///us-west-2d/i-033b9d63b15cb5420"

Checked the node, and noticed that the node now has a provider ID:

$ kubectl describe node ip-x-x-x-x.us-west-2.compute.internal 
....
ProviderID:                   aws:///us-west-2d/i-033b9d63b15cb5420
Non-terminated Pods:          (4 in total)
  Namespace                   Name                        CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                        ------------  ----------  ---------------  -------------  ---
  kube-system                 aws-node-jdh2x              25m (1%)      0 (0%)      0 (0%)           0 (0%)         5m42s
  kube-system                 coredns-799c5565b4-2cb5f    100m (5%)     0 (0%)      70Mi (1%)        170Mi (2%)     102m
  kube-system                 coredns-799c5565b4-bwb2r    100m (5%)     0 (0%)      70Mi (1%)        170Mi (2%)     102m
  kube-system                 kube-proxy-qdxm9            100m (5%)     0 (0%)      0 (0%)           0 (0%)         5m42s

Then I rolled the host back to 1.13.2 and rebooted:

bash-5.1# signpost rollback-to-inactive
bash-5.1# reboot

The host comes back fine and boots into 1.13.2, and the metadata setting-generator for provider-id is no longer there as expected:

bash-5.1# ls -al /var/lib/bottlerocket/datastore/current/live/settings/kubernetes/
total 88
drwxr-xr-x.  2 root root 4096 Apr 16 00:30 .
drwxr-xr-x. 14 root root 4096 Apr 16 00:30 ..
-rw-r--r--.  1 root root   74 Apr 16 00:30 api-server
-rw-r--r--.  1 root root    5 Apr 16 00:30 authentication-mode
-rw-r--r--.  1 root root    5 Apr 16 00:30 cloud-provider
-rw-r--r--.  1 root root 1370 Apr 16 00:30 cluster-certificate
-rw-r--r--.  1 root root   13 Apr 16 00:30 cluster-dns-ip
-rw-r--r--.  1 root root   22 Apr 16 00:30 cluster-dns-ip.setting-generator
-rw-r--r--.  1 root root   15 Apr 16 00:30 cluster-domain
-rw-r--r--.  1 root root   14 Apr 16 00:30 cluster-name
-rw-r--r--.  1 root root    2 Apr 16 00:30 max-pods
-rw-r--r--.  1 root root   16 Apr 16 00:30 max-pods.setting-generator
-rw-r--r--.  1 root root   15 Apr 16 00:30 node-ip
-rw-r--r--.  1 root root   15 Apr 16 00:30 node-ip.setting-generator
-rw-r--r--.  1 root root   71 Apr 16 00:30 pod-infra-container-image
-rw-r--r--.  1 root root   27 Apr 16 00:30 pod-infra-container-image.affected-services
-rw-r--r--.  1 root root   57 Apr 16 00:30 pod-infra-container-image.setting-generator
-rw-r--r--.  1 root root   65 Apr 16 00:30 pod-infra-container-image.template
-rw-r--r--.  1 root root   39 Apr 16 00:30 provider-id
-rw-r--r--.  1 root root    4 Apr 16 00:30 server-tls-bootstrap 
-rw-r--r--.  1 root root    5 Apr 16 00:30 standalone-mode
-rw-r--r--.  1 root root   15 Apr 16 00:30 static-pods.affected-services

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[bcressey]

LGTM if testing checks out.

[etungsten]

Push adds missing docstrings to new helper functions.

[etungsten]

Push above rebases onto develop to get changelog updates.

[etungsten]

Push above adds a new commit for updating the 1.13.3 changelog

[jpmcb]

If there's another push to be made, could update the date in the change log to 04-17

[etungsten]

Push above adjusts the date in the changelog for 1.13.3