🌱 proxy: Optionally enable token auth

[hardys]

Summary

If we want to use the front-proxy as the gateway to shard APIs, we need to enable more auth methods beyond client cert, e.g for dev/CI we probably want token auth, and for production deployments we probably want to enable OIDC.

This will allow us to reject requests which can't be authenticated before they hit any shard apiserver, and may enable future features such as rate-limiting at the proxy where we need information from authentication.

For testing the first step is to enable token auth, then we can test multiple users etc in the same way existing e2e tests do, e.g for development/CI we can now do this:

go run ./cmd/sharded-test-server --proxy-token-auth-file=$PWD/test/e2e/framework/auth-tokens.csv

In future we can easily add e.g OIDC and any other auth methods supported by the kubeapiserver code

[hardys]

/cc @sttts we discussed this recently and I added some questions, feedback welcome!

This is just a first step to enable multi-user dev-testing via the proxy, we can easily add other proxy auth methods e.g OIDC now this is refactored to use NewBuiltInAuthenticationOptions

[hardys]

/hold pending discussion re questions and e2e-sharded is not passing

[hardys]

The e2e-sharded failure is real - it's because the e2e test enables token auth at the shard, but with this PR we'll need to also enable the same token file at the front proxy (since Authorization: Bearer requests are now authenticated at the proxy, not passed through to the shard unauthenticated)

[hardys]

/retitle 🌱 proxy: Optionally enable token auth

[hardys]

/retitle [WIP] seedling proxy: Optionally enable token auth

Marking this WIP - the e2e-sharded job is still failing, but for a different reason.

The syncer in the e2e tests uses a bearer token that can't be authenticated via the token-auth-file - I don't yet fully understand where that token is generated, or how the shard API consumes it (pointers welcome!) so this is WIP while I figure that out.

[stevekuznetsov]

/assign @s-urbaniak

[hardys]

There was some discussion on slack about the e2e-sharded failure, summary:

• When the ServiceAccount auth happens in the proxy, the request UserInfo doesn't have the ClusterName when it hits the shard API after being proxied
• This results in authorizer denials because the request is considered a regular user request

I think there are two possible solutions:

1. Pass any missing UserInfo for the authorizer (in this case the ClusterName) via headers similar to the existing approach for user/groups
2. Authenticate at the proxy, but then continue to forward the request to the shard (so the request gets re-authenticated at the shard)

Currently I'm thinking (1) is preferable so we can avoid the duplicated authentication, but this is probably only workable if there are a small number of additional headers required - any opinions on this welcome! :)

[hardys]

Turns out there is already support for passing userInfo.Extra via headers in the requestheader authenticator (which the shard already enables, but we need a new arg to enable parsing the extra prefix)

That fixes the subset of tests I've been iterating on locally, but will leave this WIP until we get confirmation e2e-sharded is passing

[hardys]

/retitle :seedling proxy: Optionally enable token auth

The e2e-sharded job is now passing, so removing WIP and this is ready for review.

We'll also want to add OIDC, but given this PR already contains several commits I'm planning to propose that as a follow-up (with some details on how to test with a local test OIDC server)

One thing to mention - if any additional auth methods are enabled, we require auth at the proxy (but if neither token or service-account auth args are passed, the existing behavior is maintained).

I discussed previously that with @sttts and I'm not certain if we view that as acceptable (opt-in to the new "require auth" behavior by specifying auth methods in addition to client-cert), or if we want an explicit "require auth" flag, thoughts on that are welcome!

[hardys]

/retitle 🌱 proxy: Optionally enable token auth

[hardys]

/hold cancel

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stevekuznetsov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [stevekuznetsov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hardys]

/retitle 🌱 proxy: Optionally enable token auth