Skip to content

Commit

Permalink
Update TLS bootstrapping with 1.7 features
Browse files Browse the repository at this point in the history
This includes documenting the new CSR approver built into the
controller manager and the kubelet alpha features for certificate
rotation.

Since the CSR approver changed over the 1.7 release cycle we need
to call out the migration steps for those using the alpha feature.
This document as a whole could probably use some updates, but the
main focus of this PR is just to get these features minimally
documented before the release.

ref:

- kubernetes/kubernetes#45030
- kubernetes/enhancements#266
- kubernetes/enhancements#267
  • Loading branch information
ericchiang committed Jun 27, 2017
1 parent 7152ca3 commit 24a0436
Showing 1 changed file with 109 additions and 6 deletions.
115 changes: 109 additions & 6 deletions docs/admin/kubelet-tls-bootstrapping.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
assignees:
- ericchiang
- mikedanese
- jcbsmpsn
title: TLS bootstrapping
---

Expand Down Expand Up @@ -59,18 +61,106 @@ The kube-controller-manager flags are:
--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
```

### Automatic approval
To ease deployment and testing, there is an experimental flag in the certificate bootstrapping API to approve all certificate
requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
token in the token file above. Use of this flag circumvents the approval process described below and is not recommended
for production use.
### Approval controller

In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller
that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default.
The controller uses the [`SubjectAcessReview` API](/docs/admin/authorization/#checking-api-access) to determine
if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent
conflicts with other approvers, the builtin approver doesn't explicitly deny CSRs, only ignoring unauthorized requests.

The controller categorizes CSRs into three subresources:

1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`.
3. `selfnodeserver` - a node renewing a serving certificate.

The checks to determine if a CSR is a `selfnodeclient` or `selfnodeserver` request is currently tied to the kubelet's
credential rotation implementation, an __alpha__ feature. As such, _the definition of `selfnodeclient` and `selfnodeserver`
will likely change in a future release as the kubelet's certificate rotation graduates to beta._

The following RBAC `ClusterRoles` represent the `nodeclient`, `selfnodeclient`, and `selfnodeserver` capabilities.

```yml
# A ClusterRole which instructs the CSR approver to approve a user requesting
# node client credentials.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: approve-node-client-csr
rules:
- apiGroups: ["certificates.k8s.io"]
resources: ["certificatesigningrequests/nodeclient"]
verbs: ["create"]
---
# A ClusterRole which instructs the CSR approver to approve a node renewing its
# own client credentials.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: approve-node-client-renewal-csr
rules:
- apiGroups: ["certificates.k8s.io"]
resources: ["certificatesigningrequests/selfnodeclient"]
verbs: ["create"]
---
# A ClusterRole which instructs the CSR approver to approve a node requesting a
# serving cert matching its client cert.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: approve-node-server-renewal-csr
rules:
- apiGroups: ["certificates.k8s.io"]
resources: ["certificatesigningrequests/selfnodeserver"]
verbs: ["create"]
```

The flag is:
These powers can be granted to credentials, such as bootstrapping tokens. For example, to auto-approve all bootstrapping
tokens initial CSRs, create the following `ClusterRoleBinding`.

```yml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: auto-approve-csrs-for-group
subjects:
- kind: Group
name: system:kubelet-bootstrap
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: approve-node-client-csr
apiGroup: rbac.authorization.k8s.io
```

This `ClusterRoleBinding` is equivalent to the removed controller manager flag:

```
# REMOVED: This flag no longer works.
--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
```

To let a node renew its own credentials, an admin would construct a `ClusterRoleBinding` such as:

```yml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: node1-client-cert-renewal
subjects:
- kind: User
name: system:node:node-1 # Let "node-1" renew its client certificate.
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: approve-node-client-renewal-csr
apiGroup: rbac.authorization.k8s.io
```

Deleting the binding would prevent the node from renewing its client credentials, and effectively
remove it from the cluster once its certificate expires.

## kubelet configuration
To request a client certificate from kube-apiserver, the kubelet first needs a path to a kubeconfig file that contains the
bootstrap authentication token. You can use `kubectl config set-cluster`, `set-credentials`, and `set-context` to build this kubeconfig. Provide the name `kubelet-bootstrap` to `kubectl config set-credentials` and include `--token=<token-value>` as follows:
Expand All @@ -87,6 +177,19 @@ The flag to enable this bootstrapping when starting the kubelet is:
--experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
```

Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs.
These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature
flags on the kubelet, but may change in backward incompatible ways in future releases.

```
--feature-gates=RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true
```

`RotateKubeletClientCertificate` causes the kubelet to rotate its client certificates by creating new CSRs as its existing
credentials expire. `RotateKubeletServerCertificate` causes the kubelet to both request a serving certificate after
bootstrapping its client credentials and rotate the certificate. The serving cert currently does not request DNS or IP
SANs.

## kubectl approval
The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
Expand Down

0 comments on commit 24a0436

Please sign in to comment.