[Question or Feature Request] Make bearer token location in browser storage configurable

[mecampbellsoup]

Background

We are writing a new authentication backend server for our cloud PaaS product/platform.

If we write an OIDC-compliant auth server (i.e. our own IdP), we can configure kubeapps to use this viva oauth2-proxy for authenticating users in a more seamless way than having them copy-paste tokens. This is all well-documented e.g. here.

However, it will be much faster for us to implement a simple webhook token auth server backend instead. Moreover we aren't in love with JWTs in general and are leaning towards tried-and-true session management instead.

We intend to host kubeapps on our cloud platform's domain, e.g. cloud.coreweave.com/apps, so if we are able to write a cookie for this domain as part of normal user login flow on cloud.coreweave.com, we just need a way to tell kubeapps to read some value from a cookie and use it for bearer token auth.

I found these lines of code in the frontend apphttps://github.com/kubeapps/kubeapps/blob/9a7b52c6911e8d97f470677722043b3ae765037e/dashboard/src/shared/Auth.ts#L10-L12 whereby the kubeapps_auth_token is read out of the browser's local storage.

Current Behavior

The kubeapps frontend always looks in localStorage for the token to authenticate requests to the kubeapps backend.

Proposal

Make the choice of "in-browser token storage backend" configurable, e.g document.cookie instead of localStorage for the kubeapps_auth_token. That would be a simple and elegant solution to our use-case, I believe.

We could write a cookie containing an opaque token for that user after they login as normal. Upon navigating to /apps we can render kubeapps, and kubeapps will now be configured to read from cookies (instead of local storage, the current default token location in the browser) in order to obtain the auth token to be sent w/ requests to the backend.

[absoludity]

Hi there, just a bit of background / clarification (in case it's not obvious)

whereby the kubeapps_auth_token is read out of the browser's local storage.

This is only for demo's or similar where a K8s service account token is provided directly to Kubeapps and used for authentication. We don't recommend this for anything other than demo (nor storing any credential client side generally).

When using OIDC, we have the upstream proxy (oauth2-proxy) setting a Secure, HttpOnly cookie which the browser then includes for every request. Importantly, javascript cannot view, inspect or even detect this cookie, so there is no frontend code related to this...nor does any credential ever leave the server (the cookie can just be a session token which oauth-proxy switches for the credential on-route via redis, but in the default case, it's actually jwt token encrypted with a key known by oauth2-proxy only). Kubeapps simply checks if a request to the api server works (the browser sends the http-only cookie) and if it's 401 it redirects to the login page.

So it's not clear to me why you need any frontend code at all for your use-case? Your auth service can set whatever HttpOnly session cookie it likes (ie. a session id or encrypted webhook token) if it's happy to proxy the requests and switch it for the webhook auth token enroute. Or if your authentication provider is an oauth2 idp, you could just continue to use oauth2-proxy and have it set the webhook token as the access token (oauth2 access tokens don't have to be jwt). We already have a chart option for using the access token instead of the jwt id_token as GKE uses (a non-jwt) access token (see frontend.proxypassAccessTokenAsBearer)

Let me know if I've missed something or if that works. I've already been keen to remove the now demo-only service token support in the frontend due to the security implications, so if your use-case is that the user will enter the web hook token into the UI, I'd not be so keen - but still happy to discuss. If on the other hand it's coming from your authn service, it should be able to be kept within your servers as above (ie. client code never sees it, as we do for oidc).

Hope that helps!

[mecampbellsoup]

Thanks a lot for your extensive response! I really appreciate you taking the time.

Your auth service can set whatever HttpOnly session cookie it likes (ie. a session id or encrypted webhook token) if it's happy to proxy the requests and switch it for the webhook auth token enroute.

This sounds interesting but I'm not sure I'm following...

In my example, cloud.cw.com is/will be our "login domain" (i.e. where users login) and cloud.cw.com/apps is where we may host kubeapps (or apps.cw.com; TBD). Without using an OIDC compliant auth server is there a way we can have kubeapps talk to our auth backend, agnostic of the auth server's protocol? I would like to just write some cookie w/ e.g. sessionid=abc123-hash that can be used in conjunction with k8s webhook token auth.

Does this make sense?

[mecampbellsoup]

@absoludity in other words, if we set a cookie for domain .coreweave.com after a user logs in, and we host kubeapps at say apps.coreweave.com, will the COOKIE header be sent from kubeapps backend to our k8s webhook token auth endpoint? If so we can use that same cookie to lookup the logged-in user and perform authentication.

[absoludity]

will the COOKIE header be sent from kubeapps backend to our k8s webhook token auth endpoint?

Nope, I mean, even if the cookie header was sent, webhook authentication works with bearer tokens (Authorization: Bearer <token>) only... which I gather you've understood, so I think you're asking if the Kubeapps dashboard can pull out the content of the cookie and set it as an Authorization: Bearer <token> header for all requests to the backend/api server?

Technically yes, if you set a non httpOnly cookie, the dashboard could do that, but I would not be doing that if I were you from a security pov: it means you've got your credential for accessing the cluster being sent out to people's (insecure) browsers. For session cookies you should always be using an httpOnly cookie (which client code running on the browser cannot even access, by design). Secondly, your session cookie should either be a unique identifier only, or an encrypted credential which gets switched for the unencrypted credential by oauth2-proxy or similar once within your internal network, for similar reasons (so all that the client ever sees is a unique identifier that they can't do anything with).

Stepping back to your original question (and apologies in advance if I'm misunderstanding)

We are writing a new authentication backend server for our cloud PaaS product/platform.

I assume this is for authenticating requests to your PaaS, not for user authentication of your web app - ie. you still need users to login and be assigned some access token which can later be used with your backend authentication.

If we write an OIDC-compliant auth server (i.e. our own IdP), we can configure kubeapps to use this viva oauth2-proxy

You don't need to write the IdP used for user authentication though do you? Just use one of the many available (or your companies existing SSO, assuming it's oauth2 compliant)?

for authenticating users in a more seamless way than having them copy-paste tokens.

(yes, never do this, it's for demos only these days :) )

However, it will be much faster for us to implement a simple webhook token auth server backend instead.

So for the authentication with your cluster, yes you can either use the existing OIDC support in k8s, or create your own webhook token auth server, but either way you'll want your session cookie converted to a bearer token only once the request gets within your network. This is exactly what oauth2-proxy does (storing the credential - jwt and/or non-jwt access token - and setting a cookie then switching the cookie for the credential when requests enter the internal network).

If that's not an option, then yes, the dashboard could be updated to do what you're asking, but there's a serious security risk IMO (the same reason we only recommend the token auth for demos).

[mecampbellsoup]

@absoludity OK, thanks once again for the thoughts & feedback, it's very much appreciated 😅

You don't need to write the IdP used for user authentication though do you? Just use one of the many available (or your companies existing SSO, assuming it's oauth2 compliant)?

I assume you are referring to things like Auth0, Keycloak, etc.? We looked into these and are hesitant to go the OAuth route until we are quite sure we need it. And "company's existing SSO" - well, we are building it right now! 😆

I think what I am envisioning is a ... non-OAuth2 auth proxy? Such a proxy could e.g. check for an HttpOnly cookie header, query our sessions table to ensure the user is still logged in, and then set proxy the request with the session ID as a bearer auth token.

In other words, I am OK with trusting a session ID stored in an HttpOnly cookie (the result of user logging into cloud.coreweave.com). As long as that cookie is written to a domain like .coreweave.com then it should be sent from kubeapps frontend to this hypothetical proxy (hosted at, say, apps.coreweave.com) by default.

I believe I understand the flow I want, I am just not sure how to configure/use kubeapps to make this work... but please let me know if this is a terrible idea or if I'm missing something!

[absoludity]

@absoludity OK, thanks once again for the thoughts & feedback, it's very much appreciated sweat_smile

No problem :)

I assume you are referring to things like Auth0, Keycloak, etc.?

Or, depending whether your company uses Microsoft products or Google products, Azure Active directory, or Google authentication (or any of the myriad of providers, if your company is already using one).

We looked into these and are hesitant to go the OAuth route until we are quite sure we need it.

Hmm, you need some way of enabling your users, employees, clients to authenticate - not sure what you would use for SSO if not some OAuth identity provider.

And "company's existing SSO" - well, we are building it right now! laughing

Ok, well writing your own SSO (or implementing your own identity provider) is a pretty serious venture. Hope you've got the resources for that!

In other words, I am OK with trusting a session ID stored in an HttpOnly cookie (the result of user logging into cloud.coreweave.com). As long as that cookie is written to a domain like .coreweave.com then it should be sent from kubeapps frontend to this hypothetical proxy (hosted at, say, apps.coreweave.com) by default.

I believe I understand the flow I want, I am just not sure how to configure/use kubeapps to make this work... but please let me know if this is a terrible idea or if I'm missing something!

So if you have the rest of it sorted out, then yes, that should be trivial and not even require changes in Kubeapps, just with some nginx config, eg: https://stackoverflow.com/a/39353517 That writes an Authorization header with the value from a cookie, afaict (haven't tested). Can't say whether it's a terrible idea or not - it does still mean that the credential (even though it's httponly) is still being sent to the client unnecessarily.

[mecampbellsoup]

Or, depending whether your company uses Microsoft products or Google products, Azure Active directory, or Google authentication (or any of the myriad of providers, if your company is already using one).

FYI our use case is to expose this to customers, not developers/internal users (although really both should be supported). So while we want to give users the ability to e.g. login/signup with say GitHub or GitLab (OAuth "client" from our perspective), we're not quite convinced we need to implement OAuth in our auth server(s) and issue JWTs, etc.

Hmm, you need some way of enabling your users, employees, clients to authenticate - not sure what you would use for SSO if not some OAuth identity provider.

My understanding of SSO is that it is not inherently dependent on OAuth so, why do you say this? We want the simplest version of SSO (from Wikipedia):

A simple version of single sign-on can be achieved over IP networks using cookies but only if the sites share a common DNS parent domain.[3]

Regarding the proxy conversation...

So if you have the rest of it sorted out, then yes, that should be trivial and not even require changes in Kubeapps, just with some nginx config, eg: stackoverflow.com/a/39353517 That writes an Authorization header with the value from a cookie, afaict (haven't tested). Can't say whether it's a terrible idea or not - it does still mean that the credential (even though it's httponly) is still being sent to the client unnecessarily.

Yes, this is what I think makes sense for us, but where would this proxy sit? Between the kubeapps frontend and the kubeapps backend? Or between the kubeapps backend and our k8s API? Other? I don't see where I could insert a proxy into the kubeapps design and am feeling stupid but it's probably because it's getting quite later here ☹️

[absoludity]

FYI our use case is to expose this to customers, not developers/internal users (although really both should be supported). So while we want to give users the ability to e.g. login/signup with say GitHub or GitLab (OAuth "client" from our perspective), we're not quite convinced we need to implement OAuth in our auth server(s) and issue JWTs, etc.

Right, you might not need OAuth, but you need your users to sign in securely so that the end result is that a cookie is set on the client. At that point, sure you can roll your own, or better, use some simple but trusted package to manage logins, or use an existing (free service). I didn't mean that you can only use OAuth, I just wasn't understanding why you wouldn't.

My understanding of SSO is that it is not inherently dependent on OAuth so, why do you say this? We want the simplest version of SSO (from Wikipedia):

A simple version of single sign-on can be achieved over IP networks using cookies but only if the sites share a common DNS parent domain.[3]

Yep, I just wouldn't consider rolling my own SSO service in most cases. Sorry, I didn't mean to derail the conversation in that direction. You know your requirements :)

Regarding the proxy conversation...

So if you have the rest of it sorted out, then yes, that should be trivial and not even require changes in Kubeapps, just with some nginx config, eg: stackoverflow.com/a/39353517 That writes an Authorization header with the value from a cookie, afaict (haven't tested). Can't say whether it's a terrible idea or not - it does still mean that the credential (even though it's httponly) is still being sent to the client unnecessarily.

Yes, this is what I think makes sense for us, but where would this proxy sit? Between the kubeapps frontend and the kubeapps backend? Or between the kubeapps backend and our k8s API? Other? I don't see where I could insert a proxy into the kubeapps design and am feeling stupid but it's probably because it's getting quite later here frowning_face

Heh no, this stuff can be confusing to work through, but worth getting right :) So I don't think you need to insert a proxy as you already have one in the nginx running on the kubeapps frontend deployment. You can test it out by manually editing the frontend configmap with:

kubectl -n kubeapps edit configmap kubeapps-frontend-config

If you can get what you want working with that, happy to update the Kubeapps chat to have an option for some additional config for the frontend nginx reverse proxy.

[mecampbellsoup]

If you can get what you want working with that, happy to update the Kubeapps chat to have an option for some additional config for the frontend nginx reverse proxy.

Oh my goodness, this is really great news - I didn't realize we had an NGINX instance at our behest for this! @absoludity thank you again for taking the time, I will update you once we finalize our auth strategy here 😄 .

Yep, I just wouldn't consider rolling my own SSO service in most cases

I agree that when phrased like that it does sound daunting but, having done a bunch of research at this point, I don't see how SSO "achieved over IP networks using cookies" (to quote the Wiki again) is going to be more difficult that SSO achieved using an OAuth implementation. My stance is that we will know when OAuth/OIDC is the tool we need and it will be painfully obvious that we are missing an auth strategy that allows us to encode data in client-borne (as in "bearer") tokens that don't need to phone home to do authN/authZ.

As an example, we have a file uploader service for our VFX platform Concierge Render that is confidential (i.e. non directly user-facing) with respect to the Concierge app servers (which is a Rails app). To perform auth for file upload requests, we pass the cookie from the browser which contains a JWT that once decoded reveals the user's UID. From that we can construct the path in the file storage node (we use cephFS) and rebuild the uploaded file chunks in the right place. IMO this is a nice, clean use of JWTs. Technically since the network request is "intra-cluster" we don't even have to do TLS, nor do we have to calculate the MAC signature on the JWT (no opportunity for any MITM attack) - so this auth strategy is much faster than phoning home/hitting a DB/hitting an internal auth API endpoint, which is appropriate since we really hammer the POST /file-chunks endpoint during file transfer (we send something like 64kb data chunks, so imagine how many requests you make for a 10GB file upload...).

That said, most of our other auth needs simply won't be like this, especially as we start to fold in things like permissions (which IMO should not be sent w/ JWTs since they can change in the DB at any moment), multi-user organizations (for cloud customers signing up as companies - think like AWS/GCloud), etc.

Sorry for the diatribe but there you have it, in case you were interested in some further context.

[mecampbellsoup]

From reading the code it seems like the /api/clusters/default/ endpoint is used to determine if the user is authenticated. So in the NGINX conf I will copy this cookie ☝️ as a header Authorization: Bearer sessionid-string.

The goal is to entirely bypass the above page that prompts the user to enter a token.

[mecampbellsoup]

Hey @absoludity I'm sure you got a notification already but, just in case, I wrote some (totally not working) code to hopefully benefit this conversation and clarify what I'm going for and the changes I am proposing. 😄

#2274