-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add user accounts (Token/Certificate) #2354
Comments
I've found the following to work on Amazon Web Services with a caveat:
These files are simply extension-less JSON documents containing When the master ASG nodes come online as part of their bootup sequence they copy these files out and transform them into So, to change or add new users:
The caveat is that if a username (token file filename) contains a period it seems to cause extraneous newlines in the Hopefully it just works well for you! :) |
Thanks for the insights @r4j4h . Is there any trick I missed in order to create valid tokens? I thought this could be a perfect solution for our current authentication needs, but as soon as try to add some new tokens, the master fails to start (the new json file has a single line, no line feed, no extension in the filename, which is only edit: as it often happens, writing this cause some rubber-duck debugging effect, so here's some help if anyone else come across this and have issues with bearer token generation for kops: This generates a valid token: |
You can also use the CA capabilities now built into Kubernetes to achieve this. For example, I have a k8s v1.8 cluster running in AWS and I wanted to add another cluster admin. I generated a new RSA key and certificate signing request for the new user using openssl, making sure to set the organization to Useful commands are available here, though I opted to use |
@activeshadow you mind adding that to docs? We need to start a cluster hardening / best practices doc. |
However keep in mind that the certificates cannot be revoked, so any access you grant using those is permanent (until the cert expires). Sadly https://github.com/kubernetes/kubernetes/pull/33519 (that fixes at least parts of this) has not been applied. |
OY! Good point @vainu-arto, thanks!
Perhaps one way of getting around this would be to create new [cluster]
role binding(s) that reference a new user or group name that is used as
the common name (CN) and/or organization (O) values in the certificate
signing request?
…On Mon, Dec 11, 2017 at 11:08:59PM -0800, vainu-arto wrote:
However keep in mind that the certificates cannot be revoked, so any access you
grant using those is permanent (until the cert expires). Sadly PR #33519 (that
fixes at least parts of this) has not been applied.
|
Pending a way to manage tokens from kops, is there a convenient way to carry out this step? (We have three masters.) |
@emwalker |
@SharpEdgeMarshall thank you. I learned last night that |
no this flag just limits the command to some ig. |
These commands will reboot the master nodes and result in the updating of the
This assumes you have copied a user file to the S3 bucket as described in this comment. (Also, as has been mentioned, the token in the S3 user file must be base64 encoded.) With these commands, the masters will be unavailable while they reboot. If you stagger out the reboots, there will be a little bit of time when you may get a bad connection. There's probably a way to cordon off a master while it reboots so that the connections always work. |
@chrislovecnm which docs would you like me to add this to? |
@activeshadow you probably know better than me. Just pic a doc that makes sense or create a new one. |
@activeshadow Thanks for your advice above on adding new cluster admins through certs. I'm currently struggling to get a new user, with newly generated certs, into the 'system:masters' group. Any help you can offer to point in the right direction would be very welcome! Here's my openssl request:
When I make any kubectl command using these certs I am forbidden. If the cluster is set to allow all requests rather than rbac the new certificate authenticates and authorizes and kubectl commands work fine. But with rbac I'm forbidden with the new certs - so I'm assuming they aren't putting the user into the system:masters group somehow. Frustratingly, I can't work out a way to see what groups a user is in. Do you know if that's possible? The kubeconfig file looks like this with some info redacted, pointing to the new certificates:
Many thanks if you can help! |
Wow, a minute after I sent that I realised I was sending zero instead of O in the openssl request. |
@ollieh-m glad you got it working! FWIW, since currently certificates cannot be revoked, I would suggest avoiding putting users in the |
This apparently no longer works in kops 1.9, the secrets are not read from S3 even though "kops get secrets" lists them just fine. |
It's not working for me either. Kops 1.9.0 and cluster 1.9.6 (updated from cluster 1.8.11) |
In my understanding support for this was removed by PR #3835. |
I didn't see this coming. Maybe should be reflected in the release notes of kops 1.9 as a breaking change. I'm strugling to create certificates for the users, using @activeshadow comments, and in the meanwhile, I've done the changes manually in the .csv to keep everyone happy until I manage a problem I'm having with the certificates. Currently, the api server rejects the user's certificate I've created with an error: The steps I've followed are:
|
Ok, solved, the problem was in the Just for anyone referencing it, the script should be something like:
Just a note: feel free to remove the user group "company-users", or add new ones, just changing the line: Just take into account that those groups are tied to the certificate, and thus, not revokable without revoking the full certificate, so think it through before adding any group there. |
Just closing up: as stated before, it would be nice that the secrets change was reflected in the release notes of kops 1.9 as a breaking change. |
@vicenteampliffy glad you figured out the |
@activeshadow well, kops 1.8 populates the /srv/kubernetes/known_tokens.csv with the files stored in the s3 secrets directory, but with kops 1.9 it's no longer so, as @vainu-arto has pointed before:
|
@activeshadow All previously created user accounts in the cluster are now gone, and the method used to create them is no longer supported. Authentication needs to be redone from the ground up. I would consider it reasonable to mention this in the release notes... |
@vicenteampliffy @vainu-arto got it, thanks! So to be clear, all user accounts previously created by adding them to the I would think users added via the CSR method before the upgrade to 1.9 would still be operational, but I suspect you were adding all of them via the |
Indeed, I should have qualified that statement. "All previously created token user accounts" or something like that. For someone using token auth and not using cert auth the end result is the same, though. |
Just for completeness, I understand you may gain some of the previous functionality using this kops feature and just maintaining your known_tokens.csv file yourself: But I don't feel it's the way to go: if you mess it up you may end with a non-functional cluster, and everything seems to be pushing for Certificates. |
This really needed to go into the release notes. Rolling kops 1.9 out to our nonprod cluster and all the users are gone. Lots of moaning developers. We use tokens rather than certs so the developers can use the kube-dashboard. |
Hi @justinsb , maybe you can add this to the release notes for kops 1.9 as a breaking change? |
@corlettb I just want to reemphasize your ability to (re)grant your developers' access to your 1.9 cluster fairly easily via the CSR resources in Kubernetes. This doesn't help the fact that their access got taken away, but it can be added back pretty quickly and easily. As for giving them access to the kube-dashboard, that can be done by generating a service account for each developer (or developer groups), granting the service accounts the necessary RBAC access, and having them use the service account tokens present in the token secrets to log into the dashboard. |
Instead of using Token Auth, i would rather recommend using guard. - We use it to allow developers to access our cluster read only. - There are also some kops docs how to use it (https://github.com/appscode/guard/blob/master/docs/setup/install-kops.md). - Each github user for example gets assigned to rbac groups == github groups. This allows fine grained access. - Even static files are possible if you want. This is from my perspective the better solution as it allows you to change access rights without rolling update of your master instances. If you have any problems just ping me in the kubernetes slack #kops-user group and i will update docs to solve your problems. Greetings, Thomas |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
I would like to give access to developers to the dashboard in ReadOnly mode, i know that for RBAC i'll have to wait until 1.6 release but actually i can't even find a way to add user accounts using Token or Certificate.
Maybe i'm missing something but reading through the documentation didn't help me.
It's not clear how you should add users to the cluster with kops,
kops create secret
lets you create only a sshkey user to access the host machine.In a desperate attempt i addedd manually created certs to the pki folder in the bucket but it didn't work.
So it's possible to explain how we are supposed to add users?
The text was updated successfully, but these errors were encountered: