Use OAuth token for Kubernetes client

[ebaron]

The fabric8 Kubernetes client will use the mounted service account by default. Currently when deployed by the operator, the Cryostat pod uses the same service account used by the operator.

It would be more secure to use the provided OAuth token given by the user when authenticating. Then we know that the user will only be able to access pods that they have permission from Kubernetes to access.

I think we would also then be able to use the default service account for the Cryostat deployment when created by the operator.

[andrewazores]

Makes sense. This probably needs to be provided by an environment variable or something else similar that can be read from at startup time, or in general non-interactively, since Cryostat will need to check that token when performing actions due to automatic rules, for Kubernetes platform detection startup (adding Watches for Endpoints), etc.

[ebaron]

Right... that does complicate things a bit. Do we then need to worry about the case where a user wouldn't have permission to create a recording for a pod, but is able to do so using a rule?

[andrewazores]

We can (and I believe do) check the user's token when they create a rule definition. Although, when Cryostat goes to execute actions according to that rule later, it would do so using its own token.

[andrewazores]

Coming back around to this - I think we do need Cryostat to have a service account that it can use in "unattended" use cases, like passive background discovery of clients for automated rules. For this purpose, the service account mostly only needs to have "read"-type permissions for various resources - in Kubernetes RBAC terms, this means "watch", "list", and "get" verbs for endpoints/pods/services/etc., but not "create"/"delete"/"update" verbs for these. Symbolically it should have "* Recordings", "* FlightRecorders", "* Templates", etc. probably, too.

When a user performs an API request to Cryostat, we can (hopefully nicely - need to check how to do this nicely with the kubernetes-client) check what permissions they have. For example, if they want to create a recording, or create a rule definition which would potentially indirectly create a recording later, then their token must pass a "create Recording" RBAC check. In this case, Cryostat is operating directly on the JVM and not through the CR, so check is not enforced by Kubernetes, but by Cryostat itself. We can enumerate the various actions like this and specify what action(s), if any, an API request performs, and then pass that set to the AuthManager implementation to validate. The AuthManager would then translate that set of actions into platform-specific permissions, in this case Kubernetes RBAC, and then proceed to validate the user's token for those permissions.

Is there currently a way with Kubernetes RBAC to say that a user has permission to create a Recording only for a specific subset of FlightRecorders? Or does the user either have or not have permissions to create Recordings as a whole?

[ebaron]

Coming back around to this - I think we do need Cryostat to have a service account that it can use in "unattended" use cases, like passive background discovery of clients for automated rules. For this purpose, the service account mostly only needs to have "read"-type permissions for various resources - in Kubernetes RBAC terms, this means "watch", "list", and "get" verbs for endpoints/pods/services/etc., but not "create"/"delete"/"update" verbs for these. Symbolically it should have "* Recordings", "* FlightRecorders", "* Templates", etc. probably, too.

Makes sense. A reduced-permissions service account would definitely be an improvement.

When a user performs an API request to Cryostat, we can (hopefully nicely - need to check how to do this nicely with the kubernetes-client) check what permissions they have. For example, if they want to create a recording, or create a rule definition which would potentially indirectly create a recording later, then their token must pass a "create Recording" RBAC check. In this case, Cryostat is operating directly on the JVM and not through the CR, so check is not enforced by Kubernetes, but by Cryostat itself. We can enumerate the various actions like this and specify what action(s), if any, an API request performs, and then pass that set to the AuthManager implementation to validate. The AuthManager would then translate that set of actions into platform-specific permissions, in this case Kubernetes RBAC, and then proceed to validate the user's token for those permissions.

Sounds like a good approach to me. We effectively ensure that the user has a superset of permissions that the ServiceAccount does. I think this API will do the trick: https://kubernetes.io/docs/reference/kubernetes-api/authorization-resources/local-subject-access-review-v1/#LocalSubjectAccessReview

Is there currently a way with Kubernetes RBAC to say that a user has permission to create a Recording only for a specific subset of FlightRecorders? Or does the user either have or not have permissions to create Recordings as a whole?

Typically the user would have a RoleBinding to a Role for creating Recordings. This would allow the user to create any Recording in the namespace of the RoleBinding. This is a pretty typical use-case. It's possible to restrict the Role further to specifically named Recordings within that namespace, but this doesn't apply to the create verb, so not helpful for Recordings really [1].

[1] https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources

[andrewazores]

Okay, that's what I expected. That simplifies the permissions model quite a lot, so I think we can move forward with implementing what I've outlined pretty quickly.

Do we need to make any changes Cryostat-side to support the reduced-permissions service account? ie. have the kubernetes-client look for the token at an alternate path at startup. Or, can the operator create that reduced service account and then mount its token overtop of the standard path for Cryostat to find it at?

[ebaron]

The operator can create the service account with reduced permissions and then configure the Cryostat deployment to use it. We just need to make sure the permissions available to the service account are kept in sync with the authorization check in Cryostat.