-
Notifications
You must be signed in to change notification settings - Fork 1.6k
API server authentication to webhooks #658
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
89ea473
125c671
4a00e9a
fd79b8f
0188107
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| --- | ||
| title: API server authentication to webhooks | ||
| authors: | ||
| - "@pbarker" | ||
| - "@mattmoyer" | ||
| - "@xstevens" | ||
| owning-sig: sig-auth | ||
| participating-sigs: | ||
| - sig-api-machinery | ||
| reviewers: | ||
| - "@liggitt" | ||
| - "@tallclair" | ||
| - "@sttts" | ||
| - "@deads2k" | ||
| approvers: | ||
| - "@sttts" | ||
| - "@liggitt" | ||
| editor: TBD | ||
| creation-date: 2018-12-20 | ||
| last-updated: 2018-01-23 | ||
| status: provisional | ||
| see-also: | ||
| replaces: | ||
| superseded-by: | ||
| --- | ||
|
|
||
| # API server authentication to webhooks | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| * [API server authentication to webhooks](#api-server-authentication-to-webhooks) | ||
| * [Table of Contents](#table-of-contents) | ||
| * [Summary](#summary) | ||
| * [Motivation](#motivation) | ||
| * [Goals](#goals) | ||
| * [Non-Goals](#non-goals) | ||
| * [Proposal](#proposal) | ||
| * [User Stories](#user-stories) | ||
| * [Story 1](#story-1) | ||
| * [Story 2](#story-2) | ||
| * [Implementation Details/Notes/Constraints](#implementation-detailsnotesconstraints) | ||
| * [Risks and Mitigations](#risks-and-mitigations) | ||
| * [Graduation Criteria](#graduation-criteria) | ||
| * [Implementation History](#implementation-history) | ||
| * [Alternatives](#alternatives) | ||
|
|
||
| ## Summary | ||
|
|
||
| We want to provide a simple means of authenticating outgoing webhooks from the apiserver and its aggregates. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Outgoing webhooks from the apiserver such as the ones found in [dynamic admission control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers) and [dynamic audit](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/#dynamic-backend) | ||
| suffer from a lack of easily configurable authentication. Currently, Dynamic Admission webhooks provide a mechanism for plugin authentication by a kubeconfig provisioned on the host. The intention of this KEP is to provide a simpler means for the receiving | ||
| server to authenticate the apiserver and its aggregates using the [Authentication API](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/authentication/types.go). | ||
|
|
||
| ### Goals | ||
| * A simple means of authenticating apiserver clients. | ||
|
|
||
| ### Non-Goals | ||
| * Providing all the authentication schemes found in the current kubeconfig. | ||
|
|
||
| ## Proposal | ||
|
|
||
| We propose a simple mechanism for authenticating webhooks using the token [Authentication API](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/authentication/types.go). The shared | ||
| [webhook client](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/util/webhook/client.go) will be parameterized to optionally enhance every outgoing request with a token obtained from a | ||
| [Token Request](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/authentication/types.go#L112). The receiving server can then check that token using a [Token Review](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/authentication/types.go#L45). | ||
|
|
||
| ### User Stories | ||
|
|
||
| #### Story 1 | ||
| I am a cluster administrator using the dynamic auditing feature and want to make sure the logs I receive are from the apiserver. | ||
|
|
||
| #### Story 2 | ||
| I am a plugin developer and want to easily authenticate the apiserver. | ||
|
|
||
| ### Implementation Details/Notes/Constraints | ||
|
|
||
| A new struct will be added to the [client config](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/util/webhook/client.go#L40) for outgoing webhooks: | ||
|
|
||
| ```go | ||
| type AuthInfo struct { | ||
| ProvisionToken bool | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should model this as a union type if we think we'll add more authentication modes. |
||
| } | ||
|
|
||
| type ClientConfig struct { | ||
| Name string | ||
| URL string | ||
| CABundle []byte | ||
| AuthInfo *AuthInfo | ||
| Service *ClientConfigService | ||
| } | ||
| ``` | ||
| If enabled the client will provision a token and enhance the outgoing request with an auth header: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What service account will the kube-apiserver run as? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where does this token come from? What produces it? |
||
| ``` | ||
| Authorization: bearer <token> | ||
| ``` | ||
|
|
||
| The client will check and refresh the token when necessary. The server can now check the token using a | ||
| [TokenReview](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/authentication/types.go#L45). | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to do anything to prevent admission controllers intercepting TokenReviews from creating webhook loops? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would be nice to also allow for the tokens to use proper OIDC, if the K8S server is integrated with such a IDP. The audience-based tokens mounted in Pods seem to also support OIDC in some cases, would be great to be consistent. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with this suggestion. This can also solve the case where the webhook/audit-endpoint is not kubernetes-aware. From a quick glance, this seems to be a problem that can eventually be solved by Istio and SPIFFE. I'm guessing the author is trying to provide a better user experience that can quickly establish auth with a simple flag/config. |
||
|
|
||
| The `AuthInfo` struct will also be added to the [ClientConfig](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/auditregistration/v1alpha1/types.go#L134) in the auditregistration API. | ||
|
|
||
| The token audience would be that of the webhook name. The receiving server could verify that it is the intended | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this audience embedded in the token? If so, does the receiving Webhook extract the audience from the token, or is it relying on a review to yield a It wasn't clear on first reading, but if I understand this idea correctly, the Webhook server doesn't trust its clients, but it trusts some API server for token reviews—even though that same API server might in fact be the client of these Webhooks. Thinking about those paired connections, it seems like client certificate authentication would be easier to understand: Since the Webhook server already knows whether or not to trust the API server by way of its server certificate, the Webhook server could verify that clients that connect present a certificate signed by the same trusted CA that covers the API server's certificate. Granted, it's less practical to limit the validity period and audience scope in such X.509 certificates, but it is worth calling out why we need two network connections here to authenticate a client expected to be one and the same as a server we trust.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The webhook submits the audiences it identifies as in the TokenReviewSpec. TokenReviewStatus.auds will return all audiences that match. The webhook shouldn't need to introspect the token.
This is possible today via the webhook kubeconfig, but requires reconfiguring the kube-apiserver. I think we want to support delegated token auth in addition. I could see AuthInfo including an option that does Client TLS with a static client cert shared across all webhooks. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the identity (user name, groups, user ID) yielded by the
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (user name, groups, user ID) will not be yielded if the intersection is the empty set. If the token is compatible with the webhook (i.e. token audience and webhook API identities intersect), then it's up to the webhook to make authorization decisions based on UserInfo. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It sounds like you're assuming that the Webhook server will understand the values in the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An important use case of this feature would be to have webhooks that are shared across multiple clusters, in particular if the webhook is deployed as a cloud function or on a server with proper DNS certs ( to avoid the horrible CA-cert patching ). It would be ideal if the UserInfo would include some info about the cluster making the call, for example in GKE some reference to the project ID/zone/cluster name. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For audience, please use the base address of the URL - not the webhook name. AFAIK this is the recommended use of audience. |
||
| audience on receipt. | ||
|
|
||
| It should be noted that this solution is meant to live alongside other outgoing webhook auth solutions. For static credentials, | ||
| the existing method of [authenticating apiservers](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#authenticate-apiservers) by kubeconfig file will continue to serve that use case. The method presented is intended to ease the use of cluster aware webhooks, and can be provisioned in a dynamic manner. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Which method takes priority if both are present? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about both ? It is ok to provide client certificates ( if the server supports them ) - plus JWT/IDtoken. |
||
|
|
||
| ### Risks and Mitigations | ||
|
|
||
| Prevent server from becoming a confused deputy (making attacker-controlled call with apiserver creds). | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the mitigation here are:
Anything else?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. re @liggitt's comment above, deriving audience from the URL, rather than the webhook name, restores the desirable properties of (1). |
||
|
|
||
| ## Graduation Criteria | ||
|
|
||
| We will know if this has succeeded by telling whether it solves the majority of auth concerns around outgoing webhooks in a simple manner. | ||
|
|
||
| ## Implementation History | ||
|
|
||
| - initial draft: 12/20/2018 | ||
|
|
||
| ## Alternatives | ||
|
|
||
| We alternatively explored allowing authentication info to be provided in a secret. However, this method breaks | ||
| down in a couple scenarios. First, there is no way to differentiate between multiple API servers. This may be | ||
| sufficient in some use cases but is a security drawback in general. Next, the aggregate servers often live in | ||
| different namespaces, and there is no clear path on how a single credential could be shared between them. We | ||
| haven't abandoned this idea entirely, but feel the solution above solves the majority of use cases. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't use abbreviation in API types. How about just
Authentication?