When provisioning GKE clusters, MasterAuth has no RBAC permissions on the server.

[lukeweber]

MasterAuth Gives me a valid cert(auth) on the server, but with no RBAC(Authz).

The work-around at the moment is to use gcloud for creds to the server and then create this object:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: client-binding
subjects:
- kind: User
  name: client
roleRef:
  kind: ClusterRole
  name: "cluster-admin"
  apiGroup: rbac.authorization.k8s.io

Then all works as expected. Have I missed a config parameter, is MasterAuth deprecated. Seems like we shouldn't need to use basic auth as the only alternative and that RBAC should be applied to the generated certificate if a user requests it.

Kubernetes Master Version:
1.10.4-gke.2

[jba]

@broady @zombiezen Can you help assign this? I'm not familiar with the technologies at issue.

[jba]

@lukeweber I think this is about the API itself, not the Go client. You should probably file the bug on the appropriate k8s issue tracker.

[lukeweber]

I believe that MasterAuth should provide authz to the k8s cluster, and probably did before RBAC became standard.

The client cert that is returned by the API is Subject: CN=client. This means that when I connect to k8s with this cert I do auth but have no authorization to do anything on the server.

Basic auth that is auto generated on the other hand is the admin user for which there is a binding with full permissions.

The question/feature request would be to make this common name configurable or default to admin through the api so that generated certificates could actually be used now that RBAC is standard.

Since GKE is a private product, as well as their choices on how to configure masters and generate certs, do you have a recommendation on which kubernetes project I should post this to? This seems like the closest place to post a comment about consuming this google specific GKE resource through the API or adding configuration to so that the cert could be generated with another user like admin.'

If you want to loop someone in or maybe there's a GKE specific place where I could post this question.

[zombiezen]

As per this page, you probably want to create an issue with Kubernetes Engine.

[aeneasr]

Why is this stuff always so frustrating with Google? I literally followed a trail over several repos with no one from maintainer side wanting ownership and just delegating to the next person - and by the way delegated in the way of: "Yo, you have this problem, you fix it". Very unfortuante to see that this has not changed over the past 12 months either (different issue).

[lukeweber]

So per recommendation, I did post on the kubernetes engine bug tracker and it became this private issue:

https://issuetracker.google.com/u/1/issues/111101728, feel free to reference it, which is equivalent to kubernetes/kubernetes#65400.

In a nutshell, the client cert has CN=client encoded and client user doesn't have any permissions. If you use masterAuth username/password (basic auth), then you can apply the yaml.

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: client-binding
subjects:
- kind: User
  name: client
roleRef:
  kind: ClusterRole
  name: "cluster-admin"
  apiGroup: rbac.authorization.k8s.io

Which will give the user on the cert admin permissions. Additionally, to remove basic auth you can set the username="" in the api, but this will cause a reboot which will take 5 more minutes to do a master switch.

[aeneasr]

Thank you @lukeweber ! I'm now using your workaround, which works fine! Unfortunately, I end up with an error state in terraform after the first run due to this. I couldn't find your issue on the tracker, so I also created one (with your content in the hopes of it being picked up).

Thank you for figuring this out! It would seem pretty basic for this to work out of the box.

[lukeweber]

I updated the internal ticket just now as well. If they just change their client cert to CN=admin it would be an admin, or if they deploy with that yaml then someone could downgrade permissions in the future if they wanted. Hopefully someone reads this and just fixes it.

[jeanbza]

Sorry for the frustration, folks! I believe someone is investigating, and I've pinged just to make sure.

[aeneasr]

Thank you, appreciated

[hasheddan]

Hi @jadekler, any updates on this? Thanks for looking into it!

[jeanbza]

I've not heard any news, regrettably. cc @codyoss

[ishankhare07]

Hi, the workaround solution seems to work right now. Is there a proper solution in works on this?

[codyoss]

I have not heard anything about this at this time. I suggest trying to ping the issue referenced above to get this prioritized.

[ahmedtd]

It's not a good idea to use the client certificate in MasterAuth; this is an old bootstrapping access method. We explicitly recommend against using it in our hardening guide [1].

Note, this doesn't mean that you can't use x509 client certificates to authenticate to your cluster, you will just need to create them using the Kubernetes CSR mechanism (kubectl create csr).

However, GKE clusters are also integrated with Cloud IAM, so the easiest solution here is probably to give whatever IAM principal is executing your google-api-go-client code permissions to the cluster. For example:

gcloud container clusters add-iam-policy-binding <my-cluster> --role=roles/container.clusterAdmin --member=serviceAccount:<my-service-account>@<my-project>.iam.gserviceaccount.com

[1] https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster#restrict_authn_methods