-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[aws-eks] Can't log into fresh EKS cluster with SAML mastersRole #6982
Comments
Needs a repro |
Also see Case ID 6860089261 |
I am unable to reproduce this:
Then, execute the following command to update k8s configuration: aws eks update-kubeconfig --name <CLUSTER-NAME> --region us-east-2 --role-arn <ROLE-ARN> Then: kubectl get configmap/aws-auth -n kube-system -o yaml Returns the expected aws-auth configuration. I am closing for now. Reopen when you have additional information. |
The string vs array comment was from Amazon premium support. They have since said they were mistaken. Have you looked at the code I attached and its output of cdk synth? The steps I have to do are different and maybe that's related:
AWS said it was a problem with the config map but now they have recanted. They instructed me to open this issue. I really have no idea but the code I attached is pretty simple and does not work for the flow I described. |
What is the output you are getting when you run |
It's in the support case but effectively access denied
…On Sun, Apr 12, 2020, 12:28 PM Elad Ben-Israel ***@***.***> wrote:
What is the output you are getting when you run kubectl get all?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#6982 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABDNW6NUJVEH5G3DRPEV73RMHT3ZANCNFSM4LS5RNEA>
.
|
Can you paste the |
https://console.aws.amazon.com/support/home?region=eu-west-1#/case/?displayId=6860089261&language=en (base) ➜ kubeflow-eks git:(master) ✗ kubectl get configmap aws-auth -n kube-system -o yaml This is the CF template section generated by CDK for the awsauth: "KubeFlowClusterAwsAuthmanifest4ABE9919": { (base) ➜ kubeflow-eks git:(master) ✗ aws sts get-caller-identity (base) ➜ kubeflow-eks git:(master) ✗ aws eks update-kubeconfig --name KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3 --region eu-west-1 --role-arn arn:aws:iam::674300753731:role/CimpressADFS/vistaprint/aws-vbumodelscoring-management-team --profile vbumodelscoring-admin (base) ➜ kubeflow-eks git:(master) ✗ aws-iam-authenticator token -i KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3 Verification of the token works but yet I cannot login to EKS: To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. (base) ➜ kubeflow-eks git:(master) ✗ eksctl get cluster (base) ➜ kubeflow-eks git:(master) ✗ eksctl get iamidentitymapping --cluster KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3 (base) ➜ kubeflow-eks git:(master) ✗ eksctl get fargateprofile --cluster KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3 |
I had the same issue. I deleted the cluster and redeployed and I could log into the cluster with kubectl. |
If you have an example of a CDK stack the works with assumedroles from SAML as described in the flow here, I would be very grateful: I have tried a lot of variations and haven't found a solution other than assuming the role the stack creates for the cluster. |
@dr3s I can email you the script |
@FarshadNiayesh Would be great if you can share some details for future generations... |
@dr3s @eladb So this is the code I am using:
Is this something you were looking for? After the stack is deployed I just use the aws eks kubeconfig update command with the -r option set to the proper role. |
Thanks @FarshadNiayesh. I don't know how yours is different than what I wrote above except the role you are using. My example is specifically with using an assumed role via SAML that is already created. I'm loading it in the cdk via its ARN. You seem to be creating a role in CDK for the cluster. This should be similar to the role that's created by default and assigned to the cluster nodes. I don't have any issue assuming this role and managing the cluster, so I wouldn't expect that I would have an issue with your stack. I'll give it a try but I don't think it addresses my root issue. |
got it narrowed down. this works:
this doesn't work:
I think that it has to do with the role being SAML. I don't know why the Trusted Entities of the role would make a difference. I'll update the title of the issue to be more specific but I'm at a loss. It's possible that this has more to do with EKS than the CDK. |
Based upon experimentation, I have found it works if I do two things:
|
@dr3s this seems like an EKS problem. Can you provide us with the EKS cluster arn, so that the EKS team can investigate this further? |
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
This is still a bug and I think it is from CDK, since simply setting up the cluster with terraform with the access key tokens ect. from my SAML sso users assumed role, just works and i have console UI access to resources of the cluster without any additional role switching/assuming ect (btw terraform took me 1 week less work up till now and the cdk version still not resolved or equivalent to it :( since cdk also seems to not be able to tag existing subnets but thats another story). Using a newly created role with (and the assume policy ect from above) works:
however thats more like a hacky workaround for sso logins since in the ui console and cli switching is a bit of a nuisance and handling of long arns.... especially since its clearly possible with terraform and its eks module (or manual setup of eks). why can't cdk not simply assume the sso role correctly or just use the tokens I provide directly, this would make the use of cdk/eks with SSO extremly more simple. especially since our SSO role already was created for the purpose of EKSadmin: PS: companies seem to employ sso for aws accounts more and more and a centalized management of roles/subnets ect. this makes this even more important since the dev/deployment time gets increased immensly with this lacking support. |
I used the CDK to create an EKS cluster with an assumed role and cannot login even though I made a role that I can assume the master role. Unlike #3752 I set the mastersRole.
I followed the example here:
https://docs.aws.amazon.com/cdk/api/latest/docs/aws-eks-readme.html
Reproduction Steps
Initially I thought setting the mastersRole should be enough:
I thought that should also set up aws auth mapping in EKS but I have since added the following which also didn't help:
cluster.awsAuth.addMastersRole(clusterAdmin)
In fact this wasn't necessary and just added a duplicate master role entry but I wanted to illustrate what I tried.
Error Log
(base) ➜ kubeflow-eks git:(master) ✗ eksctl get cluster
NAME REGION
KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3 eu-west-1
(base) ➜ kubeflow-eks git:(master) ✗ eksctl get iamidentitymapping --cluster KubeFlowCluster6318BD13-370645a8943946f49942987f1352f2c3
Error: getting auth ConfigMap: Unauthorized
Environment
Other
This is the CF template section generated by CDK for the awsauth:
It may not be clear but it seems the config map isn't correct. It appears that the mapRoles array is array in a string instead of an array object.
This is 🐛 Bug Report
The text was updated successfully, but these errors were encountered: