-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Towards support for federated K8s OIDC #212
Comments
This change incorporates the OIDC token issuer into the certificates issued by Fulcio. There is a much more detailed write-up in the linked issue laying out the motivations, justifications, and long-term goals of this change. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
@bobcallaway has been working on this sort of - see #204 We use dex in front of Fulcio in prod, so the issuer is actually always the same dex endpoint. |
Yeah sorry - switched it to #204 |
This change incorporates the OIDC token issuer into the certificates issued by Fulcio. There is a much more detailed write-up in the linked issue laying out the motivations, justifications, and long-term goals of this change. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
I believe with #204 in and deployed, this work should be unblocked! |
/lgtm Talked about it offline, and it's not a blocker, but we need to figure out good UX for disambiguating and/or explicitly trusting OIDC providers during verification to prevent evil providers from minting id_tokens for valid subjects (as mentioned). Including the provider in the cert should make it fairly easy to do this with e.g. |
There are several parts to this change: 1. Implement a new `ephemeralca` that just generates an in-memory certificate, 1. Rename `pkg/ca/pkcs11ca` to `pkg/ca/x509ca` since it had nothing `PKCS11` specific (shared with `ephemeralca` logic), 1. Add support for Kubernetes OIDC via Service Account Projected Volumes, 1. Have the KinD e2e test use `ephemeralca` and `cosign sign` an image. I can split some of these pieces apart, but wanted to get this all working end-to-end, since a key goal was enabling e2e testing on KinD. This follows a lot of the ideas from: https://github.com/mattmoor/kind-oidc Related: sigstore#212 Fixes: sigstore#194 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
There are several parts to this change: 1. Implement a new `ephemeralca` that just generates an in-memory certificate, 1. Rename `pkg/ca/pkcs11ca` to `pkg/ca/x509ca` since it had nothing `PKCS11` specific (shared with `ephemeralca` logic), 1. Add support for Kubernetes OIDC via Service Account Projected Volumes, 1. Have the KinD e2e test use `ephemeralca` and `cosign sign` an image. I can split some of these pieces apart, but wanted to get this all working end-to-end, since a key goal was enabling e2e testing on KinD. This follows a lot of the ideas from: https://github.com/mattmoor/kind-oidc Related: #212 Fixes: #194 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change starts to refactor the way we interact with theh Fulcio `Config` to support a forthcoming change to support what I've taken to calling "meta URLs". I want to allow the public Fulcio instance to support the following *classes* of Issuer URL, which are exposed for EKS and GKE clusters: * https://oidc.eks.us-west-2.amazonaws.com/id/B02C93B6A2D30341AD01E1B6D48164CB * https://container.googleapis.com/v1/projects/mattmoor-credit/locations/us-west1-b/clusters/tenant-cluster To this end, I want to support adding to the Fulcio configuration "meta URLs" of the form (still a strawman): ``` https://oidc.eks.*.amazonaws.com/id/* https://container.googleapis.com/v1/projects/*/locations/*/clusters/* ``` However, in this change, I'm still just consolidating how we access the OIDC issuer information in `FulcioConfig` so that lookups are performed via a new `GetIssuer` method, and verifiers are instantiated via `GetVerifier()`. The only remaining place I see iterating over `cfg.OIDCIssuers` is for the `realm` in the `WWW-Authenticate` header of `401` responses (seems right to not include the meta URLs here). There are also still a handful of places handling OIDCIssuers directly, but most are in `config.go`, or associated validation logic (e.g. `federation/main.go`). Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change starts to refactor the way we interact with theh Fulcio `Config` to support a forthcoming change to support what I've taken to calling "meta URLs". I want to allow the public Fulcio instance to support the following *classes* of Issuer URL, which are exposed for EKS and GKE clusters: * https://oidc.eks.us-west-2.amazonaws.com/id/B02C93B6A2D30341AD01E1B6D48164CB * https://container.googleapis.com/v1/projects/mattmoor-credit/locations/us-west1-b/clusters/tenant-cluster To this end, I want to support adding to the Fulcio configuration "meta URLs" of the form (still a strawman): ``` https://oidc.eks.*.amazonaws.com/id/* https://container.googleapis.com/v1/projects/*/locations/*/clusters/* ``` However, in this change, I'm still just consolidating how we access the OIDC issuer information in `FulcioConfig` so that lookups are performed via a new `GetIssuer` method, and verifiers are instantiated via `GetVerifier()`. The only remaining place I see iterating over `cfg.OIDCIssuers` is for the `realm` in the `WWW-Authenticate` header of `401` responses (seems right to not include the meta URLs here). There are also still a handful of places handling OIDCIssuers directly, but most are in `config.go`, or associated validation logic (e.g. `federation/main.go`). Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
* Refactor the way we access `Config` This change starts to refactor the way we interact with theh Fulcio `Config` to support a forthcoming change to support what I've taken to calling "meta URLs". I want to allow the public Fulcio instance to support the following *classes* of Issuer URL, which are exposed for EKS and GKE clusters: * https://oidc.eks.us-west-2.amazonaws.com/id/B02C93B6A2D30341AD01E1B6D48164CB * https://container.googleapis.com/v1/projects/mattmoor-credit/locations/us-west1-b/clusters/tenant-cluster To this end, I want to support adding to the Fulcio configuration "meta URLs" of the form (still a strawman): ``` https://oidc.eks.*.amazonaws.com/id/* https://container.googleapis.com/v1/projects/*/locations/*/clusters/* ``` However, in this change, I'm still just consolidating how we access the OIDC issuer information in `FulcioConfig` so that lookups are performed via a new `GetIssuer` method, and verifiers are instantiated via `GetVerifier()`. The only remaining place I see iterating over `cfg.OIDCIssuers` is for the `realm` in the `WWW-Authenticate` header of `401` responses (seems right to not include the meta URLs here). There are also still a handful of places handling OIDCIssuers directly, but most are in `config.go`, or associated validation logic (e.g. `federation/main.go`). Related: #212 Signed-off-by: Matt Moore <mattomata@gmail.com> * Make `ParseConfig` private, shift verification. Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: #212 Signed-off-by: Matt Moore <mattomata@gmail.com>
These are separate from the fixed OIDC issuers, and they represent templates for *classes* of OIDC endpoints that we want to support, e.g. for EKS: ``` https://oidc.eks.*.amazonaws.com/id/* ``` The `*` character here will be used to match `[a-zA-Z0-9_-]+`, so no host or path delimiting characters are allowed to prevent attacks like: ``` https://oidc.eks.mattmoor.io/pwned.amazonaws.com/id/does-not-matter ``` We do NOT maintain OIDC verifiers for all of the possible endpoints, but we do keep an LRU cache to avoid the expensive discovery process on each request. Related: sigstore#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
So I was just able to run the following job against an EKS cluster (Graviton) and a GKE cluster (amd64):
So I think we are good here 🎉 |
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com>
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com> Signed-off-by: Billy Lynch <billy@chainguard.dev>
This change adds an ambient OIDC provider that will enable when the following filesystem path is populated: `/var/run/sigstore/cosign/oidc-token`. The intended use of this (primarily) is to enable consuming Kubernetes OIDC tokens produced through Service Account Projected Volumes. To consume this you would add the following to your Kubernetes pod spec: ```yaml containers: - name: my-container-name image: ... volumeMounts: - name: oidc-info mountPath: /var/run/sigstore/cosign volumes: - name: oidc-info projected: sources: - serviceAccountToken: path: oidc-token expirationSeconds: 600 # Use as short-lived as possible. audience: sigstore ``` This would also work with Tekton step definitions, or other things that permit the use of projected volumes. Note: Fulcio doesn't currently allow any Kubernetes provider OIDC tokens on the public instance, but one of the things I plan to look at next is supporting the endpoints from GKE and EKS (both of which have public discovery endpoints). Related: sigstore/fulcio#219 Related: sigstore/fulcio#212 Signed-off-by: Matt Moore <mattomata@gmail.com> Signed-off-by: Billy Lynch <billy@chainguard.dev>
Description
tl;dr
We should allow Fulcio to accept OIDC tokens from some (see below) federated Kubernetes OIDC issuers.Background and High-level proposal
Kubernetes has a relatively new feature called "Service Account Projected Volumes", which allows a workload to generate short-lived, audience-scoped OIDC tokens for a given serviceaccount, e.g.
At least GKE and EKS expose public issuer endpoints for these, which enable these tokens to be used with OIDC federation to authenticate with off-cluster services.
Broken assumptions and work needed
In general, the subject (
sub
) of an OIDC flow is only unique within the scoping of the issuer (iss
), but today Fulcio assumes a handful of fairly central OIDC issuers with disjoint subjects. This assumption manifests in the subject being the only piece of information included in the issued certificate. If we don't address this assumption, then we can't meaningfully add support for tokens issued by clusters because any cluster could sign things assystem:serviceaccount:default:default
!AI: So the first order of business is to make the implicit issuer URLs we have today explicit. I'd propose we start to unconditionally include the
iss
as the first URI in the URIs section of the certificate we issue.AI: Once it is safe to start including Kubernetes OIDC in the issuer configuration, I propose that we add support for Kubernetes OIDC issuers (not configured in the public instance!), and use it to stand up some basic KinD e2e testing for Fulcio following: https://github.com/mattmoor/kind-oidc. Once this lands, we can also use this same setup to start doing presubmit testing for keyless flows downstream in
cosign
(cc @dekkagaijin).AI: Once we have support for directly configured Kubernetes OIDC issuers, I propose that we refactor the current issuer configuration to support a degree of fuzzy matching, and start adding well-known public prefixes like GKE and EKS.
AI: Once we have support for Kubernetes OIDC issuers, I propose we augment the
cosign
OIDC "magic" to support a particular mount path (e.g./var/run/sigstore/cosign/oidc-token
)I believe that with the sum total of these features, we should be able to support fairly broad K8s "workload identity" based flows (both public Fulcio w/ public cloud vendor, and private Fulcios on-prem), and start supporting "keyless" presubmit testing (upstream and downstream in tools 🤩).
cc @dlorenc @lukehinds
The text was updated successfully, but these errors were encountered: