AWS: Enable EKS Pod Identity #6137
Conversation
Enable EKS Pod Identity for EKS cluster `k8s-infra-kops-prow-build`. See: https://aws.amazon.com/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/ Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Bump provider get support of EKS Pod Identity Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
k8s-ci-robot
added
cncf-cla: yes
ameukam
added
the
tide/merge-method-squash
|
What's the current platform version on the clusters? |
| @@ -53,3 +53,37 @@ resource "aws_iam_role" "google_prow_trust_role" { | |||
| max_session_duration = 43200 | |||
| assume_role_policy = data.aws_iam_policy_document.google_prow_trust_policy.json | |||
| } | |||
|
|
|||
|
|
|||
| // Leveraging EKS Pod Identity feature allow kOps prowjobs to run E2E tests | |||
There was a problem hiding this comment.
the AWS SDK of prow will need to be updated before it can use the Pod Identity credential provider logic
There was a problem hiding this comment.
@bryantbiggs Prow don't relies on AWS SDK for authentification. The AWS SDK is required for a another binary (aws-stockout) present in the test-infra repo.
There was a problem hiding this comment.
what does it use? the credential provider logic for Pod Identity is provided in the AWS SDKs
There was a problem hiding this comment.
Prow don't handle authn itself to each cloud provider we use. it only responsible to schedule each job to the kubernetes cluster. How the job interact with the cloud provider environment is up to the maintainer of the prowjob. Currently we only use access keys and secrets key for authn; they are mounted as env variables to be consumed by the job.
1.26. My understanding is that we need 1.27 at minimum to make it works. |
|
1.24 - 1.28 are supported, but they need to be on a platform version that supports pod identity. See here for the min supported platform versions for the given K8s version https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html#pod-id-cluster-versions |
Create a dedicated namespace for the prowjobs scheduled by prow.k8s.io Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
ameukam
force-pushed
the
kops-ci-eks-pod-identity
branch
from
009c555
to
a10269e
Compare
I see. Currently on |
|
/approve |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ameukam, dims The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| resource "aws_iam_role_policy_attachment" "eks_pod_identity_policy" { | ||
| provider = aws.kops-infra-ci | ||
|
|
||
| policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess" | ||
| role = aws_iam_role.eks_pod_identity_role.name | ||
| } |
There was a problem hiding this comment.
How did we pick this policy? It's privileged.
There was a problem hiding this comment.
@sftim It's intended to use a privileged account. The pod using this role need to be able to do some operations requiring Admin access. It's not possible to currently identity the entire scope of permissions needed.
There was a problem hiding this comment.
Minimum permissions required: https://kops.sigs.k8s.io/getting_started/aws/.
There was a problem hiding this comment.
If it's useful, we can craft a permissions boundary to limit what these Pods can do, including limits on elevation of those privileges. Kops does support this.
Want an issue for that?
Enable EKS Pod Identity for EKS cluster
k8s-infra-kops-prow-build.See: https://aws.amazon.com/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/
Signed-off-by: Arnaud Meukam ameukam@gmail.com