Implement TLS bootstrap for kubelet using `--bootstrap-auth-token` #30094

yifan-gu · 2016-08-04T19:38:08Z

Ref kubernetes/enhancements#43 (comment)

cc @gtank @philips @mikedanese @aaronlevy

cc @kubernetes/sig-node

Small guides on how to test this with a local cluster(thanks @gtank for help): https://gist.github.com/yifan-gu/df96e562643e49f74939fdd0bc63533a

This change is

googlebot · 2016-08-04T19:38:11Z

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

mikedanese · 2016-08-04T19:46:57Z

cc @kubernetes/sig-cluster-lifecycle

mikedanese · 2016-08-04T19:47:38Z

Thanks!

euank · 2016-08-04T19:48:04Z

cmd/kubelet/app/options/options.go

@@ -76,6 +76,11 @@ func (s *KubeletServer) AddFlags(fs *pflag.FlagSet) {
 	fs.StringVar(&s.TLSPrivateKeyFile, "tls-private-key-file", s.TLSPrivateKeyFile, "File containing x509 private key matching --tls-cert-file.")
 	fs.StringVar(&s.CertDirectory, "cert-dir", s.CertDirectory, "The directory where the TLS certs are located (by default /var/run/kubernetes). "+
 		"If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.")
+	fs.StringVar(&s.BootstrapAuthToken, "bootstrap-auth-token", s.BootstrapAuthToken, ""+
+		"A string token that is used to get the issued certificate from API server. "+
+		"If --tls-cert-file, --tls-private-key-file are provided, then kubelet will use this token get a x509 cert from the API server"+


Missing a space after server" and key."

"If --tls-cert-file and .." (assuming it really is and).

We can throw in a "respectively" at the end, though it's clear either way.

@euank will do

What if these additional flags are not specified?

Defaulting happens and they still get written to files, I had the same question #30094 (comment)

This sentence is missing a "to" between "token" and "get"

Since --tls-cert-file, --tls-private-key-file are defaulted "If --tls-cert-file, --tls-private-key-file are provided, then kubelet will" doesn't seem accurate since the kubelet will even if they aren't provided.

@mikedanese Yes, let me rephrase it.

aaronlevy · 2016-08-05T18:06:47Z

cmd/kubelet/app/server.go

+		ips = []netIP{nodeIP}
+	}
+
+	req, err := utilcertificates.NewCertificateRequest(s.TLSPrivateKeyFile, &pkix.Name{CommonName: "kubelet"}, []string{hostname}, ips)


The common name is what ends up being treated as the "user" IIRC. I also believe that if you configure --kubelet-certificate-authority in the api-server - connections to the node (by nodeName) will validate against the CN.

Maybe we should have common name default to hostname/Nodename instead?

Sorry, see that the hostname is added as SAN. But the CN="user" still stands. So I guess a question of if we would want to be able to have AuthZ on a node-by-node basis .. or if all nodes being identified by a single CN is sufficient.

I think this is an open question, but personally I'd expect all nodes to have identical (and limited) access to the API.

I think that we should go with separate users with identical access. Can we go with kubelet-$(hostname or hostname-override)? Further down we may not want identical access (e.g. we will probably want to limit access to only bound pods).

@mikedanese @gtank @euank @aaronlevy
kubelet-$(hostname/hostname-override) looks like a middle ground.

When requesting a serving cert, the CN should probably match the hostname. I don't think the CN mapping to username applies here… this won't be used as a client cert

mikedanese · 2016-08-05T18:22:32Z

I think 83494d4 should be dropped. I don't know why that's happening. It's making this PR difficult to review.

yifan-gu · 2016-08-05T19:05:03Z

I think 83494d4 should be dropped. I don't know why that's happening. It's making this PR difficult to review.

Which one? I modified the PR, the commit you referenced gets overridden now. @mikedanese

liggitt · 2016-08-18T05:13:24Z

cmd/kubelet/app/server.go

+		certPath = defaultKubeletClientCertificateFile
+	}
+	if keyPath == "" {
+		keyPath = defaultKubeletClientKeyFile


filepath.Abs(filepath.Join(certDir, "kubelet-client.key"))

k8s-github-robot · 2016-08-18T05:18:01Z

@yifan-gu PR needs rebase

liggitt · 2016-08-18T05:19:32Z

cmd/kubelet/app/server.go

+		return fmt.Errorf("unable to get cert from API server: %v", err)
+	}
+
+	// Marshal and write the kubeconfig to disk.


I think I would build a struct based on the restclient.Config the bootstrap kubeconfig resolved to, rather than including the entire bootstrap config (we don't want info from anything other than the current context, and we don't want the bootstrap credentials making it into the written kubeconfig). also, write using clientcmd.WriteToFile (see https://github.com/liggitt/kubernetes/blob/tls-bootstrap/cmd/kubelet/app/bootstrap.go#L104-L136 for struct building and marshaling)

Oh yeah, we shouldn't include the bootstrap token in the kubeconfig. But I assume the boot strap kubeconfig only has minimum context (current context).

k8s-github-robot · 2016-08-18T18:33:30Z

Labelling this PR as size/XL

k8s-bot · 2016-08-18T18:49:53Z

GCE e2e build/test failed for commit 08bcc1f59ff5bf17ba04220808c6440e531877dc.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

k8s-bot · 2016-08-18T18:54:33Z

GCE e2e build/test failed for commit 4bb4bd6.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

k8s-bot · 2016-08-18T19:09:00Z

GCE e2e build/test failed for commit 07a5c60.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

k8s-github-robot · 2016-08-18T19:59:51Z

@yifan-gu PR needs rebase

k8s-github-robot · 2016-08-18T19:59:52Z

Labelling this PR as size/L

k8s-bot · 2016-08-18T20:10:39Z

GCE e2e build/test failed for commit da67613.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

yifan-gu · 2016-08-18T20:12:46Z

Let's move dicussion to #30922 This PR page doesn't load correctly now, the commits are not the same as my tls_bootstrap branch.

k8s-bot · 2016-08-20T12:04:20Z

GCE e2e build/test failed for commit da67613.

Please reference the list of currently known flakes when examining this failure. If you request a re-test, you must reference the issue describing the flake.

@gtank

Automatic merge from submit-queue Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take) Ref kubernetes/enhancements#43 (comment) cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb Continue on the older PR #30094 as there are too many comments on that one and it's not loadable now.

errordeveloper · 2016-08-22T09:30:56Z

@yifan-gu please close this one, as #30922 has been merged.

googlebot added the cla: no label Aug 4, 2016

k8s-github-robot assigned thockin Aug 4, 2016

k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note-label-needed labels Aug 4, 2016

yifan-gu force-pushed the tls_bootstrap branch from 421c213 to 47ff607 Compare August 4, 2016 19:40

yifan-gu assigned mikedanese and unassigned thockin Aug 4, 2016

euank reviewed Aug 4, 2016
View reviewed changes

k8s-github-robot mentioned this pull request Aug 4, 2016

[k8s.io] Deployment scaled rollout deployment should not block on annotation check {Kubernetes e2e suite} #30100

Closed

k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/old-docs labels Aug 4, 2016

util/certificates: Add CSR utility function.

cf2d813

yifan-gu force-pushed the tls_bootstrap branch 2 times, most recently from 08dd84e to 83494d4 Compare August 5, 2016 17:27

aaronlevy reviewed Aug 5, 2016
View reviewed changes

yifan-gu force-pushed the tls_bootstrap branch from 83494d4 to 3f7b0aa Compare August 5, 2016 19:02

yifan-gu force-pushed the tls_bootstrap branch from 3f7b0aa to acb1048 Compare August 5, 2016 19:08

k8s-github-robot removed the kind/old-docs label Aug 5, 2016

yifan-gu force-pushed the tls_bootstrap branch 2 times, most recently from 7af7be8 to 072a86c Compare August 5, 2016 20:58

liggitt reviewed Aug 18, 2016
View reviewed changes

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 18, 2016

liggitt reviewed Aug 18, 2016
View reviewed changes

yifan-gu force-pushed the tls_bootstrap branch from 08bcc1f to 4bb4bd6 Compare August 18, 2016 18:31

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 18, 2016

k8s-github-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 18, 2016

yifan-gu force-pushed the tls_bootstrap branch from 07a5c60 to da67613 Compare August 18, 2016 19:56

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 18, 2016

k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 18, 2016

yifan-gu mentioned this pull request Aug 18, 2016

Implement TLS bootstrap for kubelet using --experimental-bootstrap-kubeconfig (2nd take) #30922

Merged

mikedanese closed this Aug 22, 2016

yifan-gu deleted the tls_bootstrap branch August 25, 2016 19:24

aaronlevy mentioned this pull request Sep 21, 2016

Refactor cert utils into one pkg, add funcs from bootkube for kubeadm to use #32876

Merged

ericchiang mentioned this pull request Apr 28, 2017

Certificate rotation for kubelet server certs. #45059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement TLS bootstrap for kubelet using `--bootstrap-auth-token` #30094

Implement TLS bootstrap for kubelet using `--bootstrap-auth-token` #30094

yifan-gu commented Aug 4, 2016 •

edited by k8s-oncall

Loading

googlebot commented Aug 4, 2016

mikedanese commented Aug 4, 2016

mikedanese commented Aug 4, 2016

euank Aug 4, 2016

yifan-gu Aug 5, 2016

vishh Aug 5, 2016

euank Aug 5, 2016

mikedanese Aug 5, 2016

mikedanese Aug 5, 2016

yifan-gu Aug 5, 2016

aaronlevy Aug 5, 2016

aaronlevy Aug 5, 2016 •

edited

Loading

gtank Aug 5, 2016

mikedanese Aug 5, 2016

yifan-gu Aug 5, 2016

liggitt Aug 8, 2016

mikedanese commented Aug 5, 2016 •

edited

Loading

yifan-gu commented Aug 5, 2016

liggitt Aug 18, 2016

k8s-github-robot commented Aug 18, 2016

liggitt Aug 18, 2016 •

edited

Loading

yifan-gu Aug 18, 2016 •

edited

Loading

k8s-github-robot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-github-robot commented Aug 18, 2016

k8s-github-robot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

yifan-gu commented Aug 18, 2016

k8s-bot commented Aug 20, 2016

errordeveloper commented Aug 22, 2016

Implement TLS bootstrap for kubelet using --bootstrap-auth-token #30094

Implement TLS bootstrap for kubelet using --bootstrap-auth-token #30094

Conversation

yifan-gu commented Aug 4, 2016 • edited by k8s-oncall Loading

googlebot commented Aug 4, 2016

mikedanese commented Aug 4, 2016

mikedanese commented Aug 4, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronlevy Aug 5, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikedanese commented Aug 5, 2016 • edited Loading

yifan-gu commented Aug 5, 2016

Choose a reason for hiding this comment

k8s-github-robot commented Aug 18, 2016

liggitt Aug 18, 2016 • edited Loading

Choose a reason for hiding this comment

yifan-gu Aug 18, 2016 • edited Loading

Choose a reason for hiding this comment

k8s-github-robot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

k8s-github-robot commented Aug 18, 2016

k8s-github-robot commented Aug 18, 2016

k8s-bot commented Aug 18, 2016

yifan-gu commented Aug 18, 2016

k8s-bot commented Aug 20, 2016

errordeveloper commented Aug 22, 2016

Implement TLS bootstrap for kubelet using `--bootstrap-auth-token` #30094

Implement TLS bootstrap for kubelet using `--bootstrap-auth-token` #30094

yifan-gu commented Aug 4, 2016 •

edited by k8s-oncall

Loading

aaronlevy Aug 5, 2016 •

edited

Loading

mikedanese commented Aug 5, 2016 •

edited

Loading

liggitt Aug 18, 2016 •

edited

Loading

yifan-gu Aug 18, 2016 •

edited

Loading