How to manage k8s secrets inside user workspaces as a cluster admin

[cccs-eric]

Summary

After reading how to mount secrets inside workspace containers, I was eager to try this out since it was the capability I was looking for to solve my problem. It works, but it seems like I might not be using this like it was expected by the Che devs. I'd like to know what is the usual way of doing the following.

My setup

I'm self-hosting Che in a restricted environment. Che runs in multi-user mode and it is configured with a che namespace for infrastructure and namespaces che-ws-<username> for user workspaces (I'm opened to change this to a single workspace for everything if need be). I'm also using a per-workspace PVC strategy. This cluster supports many users, some devs and some data scientists.

Use-case 1

As an admin, I need to configure user workspaces with the required secrets to connect to external services. For example, I have created a pyspark stack for data scientists to run Spark analytics connecting to an external Spark cluster. In order to connect to Spark, they need a shared authentication key and I would like to set that key inside Che as a k8s secret. The key is being used inside the pyspark image, so it's transparent for the data scientists. I can create a k8s secret for this with the proper annotations, but this secret is created in the che k8s namespace since it is part of infrastructure. I could have set it in the devfile, but then it would be public and that's not good. When a user creates a new workspace, the secret I created in the che workspace is not being copied to the user workspace. The documentation says

This section describes how to mount a secret from the user’s namespace as a file in single-workspace or multiple-workspace containers of Che.

so I understand it is by design, but I do not know which user will come online, so I cannot pre-create their namespace ahead of time and create the secret in there. I would have liked that for secrets like mine, Che would make a copy and create them in the user's workspace. But it isn't. What would be the recommended way of doing this (having shared secrets across users)?

Use-case 2

For my dev users, I would like to have secrets that are not shared across all of them, for example their git credentials. Those would need to go in the k8s user's namespace and it works very well. The issue I have is as an admin, how do you create those secrets? The user doesn't have access to the k8s cluster, so how do they create those secrets? I do not know their credentials (username/password), so I cannot create the git-credential secret on their behalf. What is the common way of doing this? How do admins manage the security aspect of this kind of situation (user secrets)?

What am I looking to get out of this?

I don't expect a clear answer like "simply do this and you are done". Please educate me on how to manage Che wisely to have a nice user experience 😃

[yhontyk]

https://issues.redhat.com/browse/RHDEVDOCS-2670

[rkratky]

@l0rd, could you please help to shed some light on this? Are the described use cases valid? Or, to put it better, are there ways to work around those problems, or is it a shortcoming of the app?

In other words, should this be fixed in the app, or should docs be written about it?

[l0rd]

@rkratky that's not a documentation issue.

@cccs-eric is asking for 2 new features:

FEATURE #1 Allow configuring workspaces secret shared across users
We don't have anything planned yet but may be interesting to implement in the future cc @sympatheticmoose

FEATURE #2 Users should be able to mange workspaces secrets via the UI (Dashboard/Theia)
That's something that we are already working on c.f. #17954. @cccs-eric you may want to have a look and comment if that would address your second use case. If users do not have access to Kubernetes there is no workaround I am aware of right now. In general Che works better if Che users are Kubernetes users too so that from the Che terminal they have access to the kubernetes API (to manage secrets but also to deploy the application they are developing etc...).

[sympatheticmoose]

wondering whether specifying a SealedSecret in the devfile would help with use-case1... not something we currently support (https://www.eclipse.org/che/docs/che-7/end-user-guide/configuring-a-workspace-using-a-devfile/#objects-supported-in-che_che) but more generally we could look at applying k8s components of 'unknown' kind. Will start a new issue to discuss that

[cccs-eric]

@sympatheticmoose Sorry for resurrecting an old issue here, but I managed to get around my use-case #1 by having all my user workspaces located in the same k8s namespace. But it is my understanding that Che is moving away from supporting this mode and prefers to have each user in its own k8s namespace. Am I right?

Assuming I am, I understand that having a secret shared across every workspace might not make sense to everyone. How can a Quarkus workspace share anything with a C/C++ workspace, right??? We can find a use-case for anything, but let's not go crazy. You mentioned something about a SealedSecret which sounds interesting. I couldn't find information about this though. I think this is something that would be defined in a devfile?

If the idea of having secrets defined in the che namespace (the namespace where che-server, keycloak, etc.. are deployed) is something that makes sense for Che, I think it would be relatively easy to implement with an annotation on a secret. In my organization, assuming that users have access to the Kubernetes API is not something we want to entertain for various reasons.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[l0rd]

/remove-lifecycle stale

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.