-
Notifications
You must be signed in to change notification settings - Fork 793
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS: Enable EKS Pod Identity #6137
AWS: Enable EKS Pod Identity #6137
Conversation
Enable EKS Pod Identity for EKS cluster `k8s-infra-kops-prow-build`. See: https://aws.amazon.com/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/ Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Bump provider get support of EKS Pod Identity Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
What's the current platform version on the clusters? |
@@ -53,3 +53,37 @@ resource "aws_iam_role" "google_prow_trust_role" { | |||
max_session_duration = 43200 | |||
assume_role_policy = data.aws_iam_policy_document.google_prow_trust_policy.json | |||
} | |||
|
|||
|
|||
// Leveraging EKS Pod Identity feature allow kOps prowjobs to run E2E tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the AWS SDK of prow will need to be updated before it can use the Pod Identity credential provider logic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bryantbiggs Prow don't relies on AWS SDK for authentification. The AWS SDK is required for a another binary (aws-stockout) present in the test-infra repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does it use? the credential provider logic for Pod Identity is provided in the AWS SDKs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prow don't handle authn itself to each cloud provider we use. it only responsible to schedule each job to the kubernetes cluster. How the job interact with the cloud provider environment is up to the maintainer of the prowjob. Currently we only use access keys and secrets key for authn; they are mounted as env variables to be consumed by the job.
1.26. My understanding is that we need 1.27 at minimum to make it works. |
1.24 - 1.28 are supported, but they need to be on a platform version that supports pod identity. See here for the min supported platform versions for the given K8s version https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html#pod-id-cluster-versions |
Create a dedicated namespace for the prowjobs scheduled by prow.k8s.io Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
009c555
to
a10269e
Compare
I see. Currently on |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ameukam, dims The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
resource "aws_iam_role_policy_attachment" "eks_pod_identity_policy" { | ||
provider = aws.kops-infra-ci | ||
|
||
policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess" | ||
role = aws_iam_role.eks_pod_identity_role.name | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How did we pick this policy? It's privileged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sftim It's intended to use a privileged account. The pod using this role need to be able to do some operations requiring Admin access. It's not possible to currently identity the entire scope of permissions needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minimum permissions required: https://kops.sigs.k8s.io/getting_started/aws/.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it's useful, we can craft a permissions boundary to limit what these Pods can do, including limits on elevation of those privileges. Kops does support this.
Want an issue for that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. And thank you!
Enable EKS Pod Identity for EKS cluster
k8s-infra-kops-prow-build
.See: https://aws.amazon.com/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/
Signed-off-by: Arnaud Meukam ameukam@gmail.com