node authorizer sets up access rules for dynamic config #60100

mtaufen · 2018-02-20T22:27:20Z

This PR makes the node authorizer automatically set up access rules for
dynamic Kubelet config.

I also added some validation to the node strategy, which I discovered we
were missing while writing this.

This PR is based on another WIP from @liggitt.

The node authorizer now automatically sets up rules for Node.Spec.ConfigSource when the DynamicKubeletConfig feature gate is enabled.

mtaufen · 2018-02-26T20:00:35Z

I sent pings trying to get this approved for 1.10 milestone, if it doesn't, so be it.

liggitt · 2018-03-22T20:21:56Z

plugin/pkg/auth/authorizer/node/graph_populator.go

+
+	path := "nil"
+	if node.Spec.ConfigSource != nil {
+		path = fmt.Sprintf("/api/v1/namespaces/%s/configmaps/%s", namespace, name)


just log %s/%s namespace/name, no need to make the API path

liggitt · 2018-03-22T20:39:27Z

add a positive and negative test case to the auth/node integration test as well

mtaufen · 2018-03-22T23:38:34Z

pkg/apis/core/validation/validation.go

+	if ref := source.ConfigMapRef; ref != nil {
+		count++
+		// name, namespace, and UID must all be non-empty for ConfigMapRef
+		if ref.Name == "" || ref.Namespace == "" || string(ref.UID) == "" {


This has been a requirement of dynamic Kubelet config since the original proposal, to require users to be completely explicit (time and space) about which ConfigMap to use. If the UID in the config source doesn't match the UID of the ConfigMap, the Kubelet currently rejects it.

IIRC there were concerns along the lines of ConfigMaps being deleted and recreated with the same name but different content... though this is less of an issue today now that we have the --append-hash option in kubectl (obviously the potential for in-place ConfigMap mutation still exists, but if users are using --append-hash I wouldn't expect them to mutate the ConfigMap after creation).

The Kubelet also checkpoints by UID, so it was convenient to already have this specified in the object ref, though I guess if we relaxed this requirement it could just resolve the UID and stick it in the local copy of the config source when it first downloads the ConfigMap.

I can't remember other reasons to require UID, if any.

We should probably discuss this in parallel to this PR; if we relax the UID requirement there is other code that needs to change too, and I'd like to make all those changes in a single, isolated PR. I'm glad you brought this up; having this discussion prior to DKC beta was on my todo list.

mtaufen · 2018-03-22T23:57:05Z

plugin/pkg/auth/authorizer/node/graph_populator.go

+
+	path := "nil"
+	if node.Spec.ConfigSource != nil {
+		path = fmt.Sprintf("/api/v1/namespaces/%s/configmaps/%s", namespace, name)


mtaufen · 2018-03-22T23:58:03Z

plugin/pkg/auth/authorizer/node/graph.go

+	defer g.lock.Unlock()
+
+	// nothing to do if we don't have a node
+	if len(nodeName) > 0 {


I guess I wouldn't expect us to get a node with an empty name, but would we catch such a case earlier than this?

mtaufen · 2018-03-23T00:00:01Z

plugin/pkg/auth/authorizer/node/graph.go

+// must be called under write lock
+// deletes edges from a given vertex type to a given vertex type
+// will also delete the "from" nodes if they are orphaned, but will not delete the "to" node
+func (g *Graph) deleteEdges_locked(fromType, toType vertexType, toNamespace, toName string) {


agree wrt tests, will add
pretty much indifferent on whether we make this a generic graph method

mtaufen · 2018-03-23T00:05:13Z

plugin/pkg/auth/authorizer/node/graph.go

+		g.deleteEdges_locked(configMapVertexType, nodeVertexType, "", nodeName)
+
+		// establish new edges if we have a real ConfigMap to reference
+		if len(configMapName) > 0 {


note to self, also handle empty namespace?

done, now we also skip if the namespace is an empty string

mtaufen · 2018-03-23T00:09:37Z

plugin/pkg/auth/authorizer/node/graph.go

+func (g *Graph) DeleteNode(nodeName string) {
+	g.lock.Lock()
+	defer g.lock.Unlock()
+	if len(nodeName) > 0 {


this check probably also unnecessary wrt @liggitt's comment above

I think this whole function is not needed (and was weird anyway, since it didn't actually delete the node)

mtaufen · 2018-03-23T00:12:19Z

plugin/pkg/auth/authorizer/node/graph_populator.go

+
+func (g *graphPopulator) deleteNode(obj interface{}) {
+	node := obj.(*api.Node)
+	g.graph.DeleteNode(node.Name)


Hmm, since we don't actually delete the Node from the graph anyway, that sounds like it'd work.

mtaufen

Updated, still have to add the integration test cases.

mtaufen · 2018-03-23T20:25:39Z

plugin/pkg/auth/authorizer/node/graph_populator.go

+
+	path := "nil"
+	if node.Spec.ConfigSource != nil {
+		path = fmt.Sprintf("/api/v1/namespaces/%s/configmaps/%s", namespace, name)


mtaufen · 2018-03-23T20:31:59Z

plugin/pkg/auth/authorizer/node/graph.go

+	defer g.lock.Unlock()
+
+	// nothing to do if we don't have a node
+	if len(nodeName) > 0 {


There's a similar len(nodeName) check in this file in AddVolumeAttachment, is there a reason for this one?

mtaufen · 2018-03-23T20:43:55Z

plugin/pkg/auth/authorizer/node/graph.go

+		g.deleteEdges_locked(configMapVertexType, nodeVertexType, "", nodeName)
+
+		// establish new edges if we have a real ConfigMap to reference
+		if len(configMapName) > 0 {


done, now we also skip if the namespace is an empty string

mtaufen · 2018-03-23T22:17:43Z

plugin/pkg/auth/authorizer/node/graph.go

+func (g *Graph) DeleteNode(nodeName string) {
+	g.lock.Lock()
+	defer g.lock.Unlock()
+	if len(nodeName) > 0 {


mtaufen · 2018-03-23T22:17:57Z

plugin/pkg/auth/authorizer/node/graph_populator.go

+
+func (g *graphPopulator) deleteNode(obj interface{}) {
+	node := obj.(*api.Node)
+	g.graph.DeleteNode(node.Name)


mtaufen · 2018-03-23T22:22:35Z

plugin/pkg/auth/authorizer/node/graph.go

+// must be called under write lock
+// deletes edges from a given vertex type to a given vertex type
+// will also delete the "from" nodes if they are orphaned, but will not delete the "to" node
+func (g *Graph) deleteEdges_locked(fromType, toType vertexType, toNamespace, toName string) {


added some test cases

This PR makes the node authorizer automatically set up access rules for dynamic Kubelet config. I also added some validation to the node strategy, which I discovered we were missing while writing this.

mtaufen · 2018-03-29T18:52:08Z

/retest

liggitt · 2018-03-29T23:17:42Z

test/integration/auth/node_test.go

+					Name:      "myconfigmapconfigsource",
+					// validation just requires UID to be non-empty and it isn't necessary for GET,
+					// so we just use a bogus one for the test
+					UID: "uid",


if you have valid data from creating the configmap above, go ahead and set it here

liggitt · 2018-03-29T23:19:29Z

/lgtm

k8s-ci-robot · 2018-03-29T23:20:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, mtaufen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/apis/core/OWNERS~~ [liggitt]
~~pkg/kubeapiserver/OWNERS~~ [liggitt]
~~plugin/pkg/auth/authorizer/node/OWNERS~~ [liggitt]
~~test/OWNERS~~ [liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-github-robot · 2018-03-30T00:37:16Z

Automatic merge from submit-queue (batch tested with PRs 61829, 61908, 61307, 61872, 60100). If you want to cherry-pick this change to another branch, please follow the instructions here.

mtaufen assigned liggitt Feb 20, 2018

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 20, 2018

k8s-ci-robot requested review from deads2k, ericchiang and nikhiljindal February 20, 2018 22:27

mtaufen force-pushed the node-authz-nodeconfigsource branch 5 times, most recently from f708dfd to fa553c1 Compare February 21, 2018 23:56

mtaufen mentioned this pull request Feb 21, 2018

Enable node authorizer in e2e node tests #60172

Closed

mtaufen changed the title ~~(WIP) node authorizer sets up access rules for dynamic config~~ node authorizer sets up access rules for dynamic config Feb 22, 2018

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2018

mtaufen removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 22, 2018

mtaufen force-pushed the node-authz-nodeconfigsource branch from fa553c1 to 2c870de Compare February 22, 2018 01:24

mtaufen modified the milestones: next-candidate, v1.10 Feb 26, 2018

mtaufen mentioned this pull request Feb 27, 2018

Dynamic Kubelet Configuration kubernetes/enhancements#281

Closed

27 tasks

k8s-github-robot added the milestone/incomplete-labels label Mar 2, 2018

liggitt reviewed Mar 22, 2018

View reviewed changes

mtaufen commented Mar 23, 2018

View reviewed changes

mtaufen force-pushed the node-authz-nodeconfigsource branch from 2c870de to 1311235 Compare March 23, 2018 22:23

mtaufen commented Mar 23, 2018

View reviewed changes

mtaufen force-pushed the node-authz-nodeconfigsource branch 3 times, most recently from 56e65a4 to 8069d89 Compare March 23, 2018 23:23

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 23, 2018

mtaufen force-pushed the node-authz-nodeconfigsource branch 7 times, most recently from 8e6fdcb to 105e546 Compare March 24, 2018 03:30

mtaufen mentioned this pull request Mar 25, 2018

Determine whether we should continue to require UID in node config source reference #61643

Closed

mtaufen force-pushed the node-authz-nodeconfigsource branch 2 times, most recently from 4ef3454 to 7bb2e74 Compare March 27, 2018 15:48

node authorizer sets up access rules for dynamic config

ab8dc12

This PR makes the node authorizer automatically set up access rules for dynamic Kubelet config. I also added some validation to the node strategy, which I discovered we were missing while writing this.

mtaufen force-pushed the node-authz-nodeconfigsource branch from 7bb2e74 to ab8dc12 Compare March 27, 2018 15:49

liggitt reviewed Mar 29, 2018

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 29, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 29, 2018

k8s-github-robot merged commit 5ae7bba into kubernetes:master Mar 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

node authorizer sets up access rules for dynamic config #60100

node authorizer sets up access rules for dynamic config #60100

mtaufen commented Feb 20, 2018

mtaufen commented Feb 26, 2018

liggitt Mar 22, 2018

mtaufen Mar 22, 2018

mtaufen Mar 23, 2018

liggitt commented Mar 22, 2018

mtaufen Mar 22, 2018

mtaufen Mar 22, 2018

mtaufen Mar 22, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

liggitt Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen left a comment

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen Mar 23, 2018

mtaufen commented Mar 29, 2018

liggitt Mar 29, 2018

liggitt commented Mar 29, 2018

k8s-ci-robot commented Mar 29, 2018

k8s-github-robot commented Mar 30, 2018

node authorizer sets up access rules for dynamic config #60100

node authorizer sets up access rules for dynamic config #60100

Conversation

mtaufen commented Feb 20, 2018

mtaufen commented Feb 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liggitt commented Mar 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtaufen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtaufen commented Mar 29, 2018

Choose a reason for hiding this comment

liggitt commented Mar 29, 2018

k8s-ci-robot commented Mar 29, 2018

k8s-github-robot commented Mar 30, 2018