☂️ Projected Service Account Tokens

[rfranzke]

How to categorize this issue?

/area security
/kind enhancement
/priority 3

This is an epic to track the development related to leveraging projected ServiceAccount tokens for the components Gardener deploys into seed and shoot clusters

Merge gardener/gardener-resource-manager repository into gardener/gardener since we decided to add new controllers/webhooks to GRM as it's already deployed for all seeds and shoots:

• Move github.com/gardener/gardener-resource-manager into gardener/gardener #4757
• Deploy gardener-resource-manager in the same version like gardenlet #4848
• Fix resource-manager image tag #4871

Mimic upstream rootcacertpublisher controller in kube-controller-manager to make its functionality available for old Kubernetes versions

• Mimic k/k's upstream rootcacertpublisher controller #4832
• Enablement of this new controller for seed and shoot clusters (Enable RootCAPublisher and make gardener-resource-manager use a projected ServiceAccount token #4940)

Webhook for automatically adding projected ServiceAccount token volumes and volumeMounts for Pods (so that we don't have to manually adapt all the components deployed in seeds and shoots)

• Add webhook handler for auto-mounting projected ServiceAccount tokens into Pods #4873
• Ensure gardener-resource-manager is highly available #4941
• Enablement of this new webhook for seed and shoot clusters (Enable gardener-resource-manager's TokenInvalidator and ProjectedTokenMount webhooks in seed and shoot #5002)
• Replace usage of static ServiceAccount tokens for components running in seed talking to the seed API server #4659
• Replace usage of static ServiceAccount tokens for components running in shoot talking to the shoot API server #4660

Controller for creating ServiceAccounts in shoot clusters, requesting short-lived tokens, and pushing them back into the shoot namespaces in the seed for the control plane components to use (e.g. kube-scheduler) when communicating with the shoots' kube-apiservers

• Add ServiceAccount token requestor controller #4867
• Enablement of this new controller for shoot clusters (Enable TokenRequestor and eliminate client certificate for kube-scheduler #4931)
• Replace usage of static client credentials for components running in seed talking to the shoot API server #4661

Controller for invalidating static ServiceAccount tokens since we plan to not use them anymore and since Kubernetes does not provide such a feature yet

• Add static ServiceAccount token invalidator #4817
• Enablement of this new controller and webhook for seed and shoot clusters (Enable gardener-resource-manager's TokenInvalidator and ProjectedTokenMount webhooks in seed and shoot #5002)

Miscellaneous

• 🛡 Provide some helpful log messages for TokenInvalidator and TokenRequestor #5341
• 🛡 Provide some helpful log messages for ProjectedTokenMount #5506
• 🛡 Invalidate ServiceAccounts for controllers part of kube-controller-manager #5422
  • The unused static ServiceAccount tokens for the controllers part of kube-controller-manager in the kube-system namespace of shoot clusters are now invalidated for all Kubernetes versions. However, note that the tokens for three of these controllers ({node,route,service} controllers) will only be invalidated for Kubernetes 1.21+ clusters since the cloud-controller-managers of prior versions still rely on them.

Related, but out of scope for now and potential improvements for the future

• Consider whether we want to consciously protect the ServiceAccounts in the shoot created by the TokenRequestor (ref Protect Service Accounts Against Deletion/Remedy Deletions #55)
• Make TokenInvalidator and ProjectedVolumeMount usable for shoot cluster users #4924
• Add metrics and Grafana dashboards for Token{Requestor,Invalidator} controllers part of gardener-resource-manager

[rfranzke]

All related tasks have been completed.
/close

[gardener-prow]

@rfranzke: Closing this issue.

In response to this:

All related tasks have been completed.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.