Update TLS bootstrapping with 1.7 features #4208

ericchiang · 2017-06-27T21:58:06Z

This includes documenting the new CSR approver built into the
controller manager and the kubelet alpha features for certificate
rotation.

Since the CSR approver changed over the 1.7 release cycle we need
to call out the migration steps for those using the alpha feature.
This document as a whole could probably use some updates, but the
main focus of this PR is just to get these features minimally
documented before the release.

ref:

cc @mikedanese @jcbsmpsn @luxas @liggitt

This change is

jcbsmpsn · 2017-06-27T22:30:50Z

docs/admin/kubelet-tls-bootstrapping.md

+The controller categorizes CSRs into three subresources:
+
+1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
+2. `selfnodeclient` - a node re-requesting a client certificate with the same `O` and `CN`.


s/re-requesting/renewing/

jcbsmpsn · 2017-06-27T22:31:54Z

docs/admin/kubelet-tls-bootstrapping.md

+
+1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
+2. `selfnodeclient` - a node re-requesting a client certificate with the same `O` and `CN`.
+3. `selfnodeserver` - a node requesting or re-requesting a serving certificate.


s/re-requesting/renewing/

Why include 'requesting'?

To indicate that this is for the initial serving cert too. Will change.

jcbsmpsn · 2017-06-27T22:34:43Z

docs/admin/kubelet-tls-bootstrapping.md

+2. `selfnodeclient` - a node re-requesting a client certificate with the same `O` and `CN`.
+3. `selfnodeserver` - a node requesting or re-requesting a serving certificate.
+
+The checks to determine if a CSR is a `selfndeclient` or `selfnodeserver` request is currently tied to the kubelet's


s/selfndeclient/selfnodeclient/

jcbsmpsn · 2017-06-27T22:53:34Z

docs/admin/kubelet-tls-bootstrapping.md

@@ -87,6 +177,19 @@ The flag to enable this bootstrapping when starting the kubelet is:
 --experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
 ```

+Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs.
+These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature
+flags on the kubelet, but may change in backward incompatible ways in future releases.


Also RotateKubeletServerCertificate on the API server. The auto approver for the server certificate is feature gated.

What CSR logic is in the API server? I would have expected that in the controller manager. Does the controller manager detect this based on the API server configuration?

Sorry, I mis-spoke. It is indeed in the controller manager.

Thanks for clarifying. Will update the doc.

chenopis · 2017-06-27T22:57:13Z

BTW, I think we should turn this into a Task, which I can do in a follow up PR. Currently, it's not findable in the table of contents.

luxas

Just some nits from tech review

luxas · 2017-06-28T09:43:30Z

docs/admin/kubelet-tls-bootstrapping.md


-The flag is:
+In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller


typo: experimental
would s/of a new/of the new/ make sense?

luxas · 2017-06-28T09:44:03Z

docs/admin/kubelet-tls-bootstrapping.md


-The flag is:
+In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller
+that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default.


suggest: enabled by default

luxas · 2017-06-28T09:44:16Z

docs/admin/kubelet-tls-bootstrapping.md

-The flag is:
+In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller
+that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default.
+The controller uses the [`SubjectAcessReview` API](/docs/admin/authorization/#checking-api-access) to determine


SubjectAccessReview

luxas · 2017-06-28T09:45:14Z

docs/admin/kubelet-tls-bootstrapping.md

+3. `selfnodeserver` - a node renewing a serving certificate. (ALPHA, requires feature gate)
+
+The checks to determine if a CSR is a `selfnodeserver` request is currently tied to the kubelet's credential rotation
+implementation, an __alpha__ feature. As such, the definition of `selfnodeserver` will likely change in a future and


link to the features issue?

luxas · 2017-06-28T09:46:26Z

docs/admin/kubelet-tls-bootstrapping.md

+# A ClusterRole which instructs the CSR approver to approve a user requesting
+# node client credentials.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1


Are there auto-bootstrapped ClusterRoles for these?
I think that would make a lot of sense, at least in v1.8

luxas · 2017-06-28T09:47:08Z

docs/admin/kubelet-tls-bootstrapping.md

+  name: auto-approve-csrs-for-group
+subjects:
+- kind: Group
+  name: system:kubelet-bootstrap


What kind of group is this? Is it well-known or just something you wrote here?

This was trying to provide a migration path for users who where using the deprecated approver. Will update the docs, thanks.

--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"

luxas · 2017-06-28T09:48:17Z

docs/admin/kubelet-tls-bootstrapping.md

 --insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
 ```

+To let a node renew its own credentials, an admin would construct a `ClusterRoleBinding` such as:


Is it worth showing how you can let all nodes in the cluster renew their CSRs automatically by binding to the system:nodes group or should the user just understand that?

I think it'd be best practice to have each node get a role binding so you can revoke individual node access, so I'd rather avoid suggesting that.

luxas · 2017-06-28T09:49:46Z

docs/admin/kubelet-tls-bootstrapping.md

+The controller categorizes CSRs into three subresources:
+
+1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
+2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`.


I think this is pretty much alpha as well @jcbsmpsn, right?

This is "beta" and will hopefully not change. selfnodeserver is the only recognizer that is alpha and will change

Ok, cool, so it's just kubelet's mechanism of renewing those client certs that is alpha. Gotcha.

This includes documenting the new CSR approver built into the controller manager and the kubelet alpha features for certificate rotation. Since the CSR approver changed over the 1.7 release cycle we need to call out the migration steps for those using the alpha feature. This document as a whole could probably use some updates, but the main focus of this PR is just to get these features minimally documented before the release.

ericchiang · 2017-06-28T18:57:09Z

Thanks everyone for the reviews. I think this is good to go.

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 27, 2017

k8s-github-robot assigned mikedanese Jun 27, 2017

chenopis added this to the 1.7 milestone Jun 27, 2017

chenopis self-assigned this Jun 27, 2017

chenopis added Needs Docs Review labels Jun 27, 2017

chenopis requested review from liggitt, mikedanese, luxas and jcbsmpsn June 27, 2017 22:12

chenopis unassigned mikedanese Jun 27, 2017

ericchiang force-pushed the tls-boostrapping-17 branch from f6ed129 to 4b88f61 Compare June 27, 2017 22:15

jcbsmpsn reviewed Jun 27, 2017

View reviewed changes

jcbsmpsn mentioned this pull request Jun 27, 2017

Kubelet Server TLS Certificate Rotation kubernetes/enhancements#267

Open

ericchiang force-pushed the tls-boostrapping-17 branch from 4b88f61 to 24a0436 Compare June 27, 2017 22:43

chenopis added Tech Review: Open Issues and removed Needs Tech Review labels Jun 27, 2017

jcbsmpsn reviewed Jun 27, 2017

View reviewed changes

chenopis added Docs LGTM and removed Needs Docs Review labels Jun 27, 2017

ericchiang force-pushed the tls-boostrapping-17 branch from 24a0436 to 6bc0bb9 Compare June 28, 2017 00:23

luxas reviewed Jun 28, 2017

View reviewed changes

luxas mentioned this pull request Jun 28, 2017

Add ClusterRoles for CSR approval modes to RBAC bootstrappolicy kubernetes/kubernetes#48191

Closed

ericchiang force-pushed the tls-boostrapping-17 branch from 6bc0bb9 to fbb1904 Compare June 28, 2017 18:56

chenopis added the Tech Review LGTM label Jun 28, 2017

chenopis removed the Tech Review: Open Issues label Jun 28, 2017

chenopis merged commit 93861d8 into kubernetes:release-1.7 Jun 28, 2017

ericchiang deleted the tls-boostrapping-17 branch June 28, 2017 19:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update TLS bootstrapping with 1.7 features #4208

Update TLS bootstrapping with 1.7 features #4208

ericchiang commented Jun 27, 2017 •

edited

jcbsmpsn Jun 27, 2017

jcbsmpsn Jun 27, 2017

ericchiang Jun 27, 2017

jcbsmpsn Jun 27, 2017

jcbsmpsn Jun 27, 2017

ericchiang Jun 27, 2017

jcbsmpsn Jun 28, 2017

ericchiang Jun 28, 2017

chenopis commented Jun 27, 2017

luxas left a comment

luxas Jun 28, 2017

luxas Jun 28, 2017

luxas Jun 28, 2017

luxas Jun 28, 2017

luxas Jun 28, 2017

luxas Jun 28, 2017

ericchiang Jun 28, 2017

luxas Jun 28, 2017

ericchiang Jun 28, 2017

luxas Jun 28, 2017

mikedanese Jun 28, 2017

luxas Jun 28, 2017

ericchiang commented Jun 28, 2017


		The flag is:
		In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller

Update TLS bootstrapping with 1.7 features #4208

Update TLS bootstrapping with 1.7 features #4208

Conversation

ericchiang commented Jun 27, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chenopis commented Jun 27, 2017

luxas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericchiang commented Jun 28, 2017

ericchiang commented Jun 27, 2017 •

edited