New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit node access to API #279

Open
saad-ali opened this Issue Apr 26, 2017 · 23 comments

Comments

Projects
@saad-ali
Member

saad-ali commented Apr 26, 2017

Limit node access to API

  • One-line feature description (can be used as a release note):
    • A new Node authorization mode and NodeRestriction admission plugin, when used in combination, limit nodes' access to specific APIs, so that they may only modify their own Node API object, only modify Pod objects bound to themselves, and only retrieve secrets and configmaps referenced by pods bound to themselves.
  • Primary contact (assignee):
  • Responsible SIGs:
    • sig/auth
  • Design proposal link (community repo):
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
  • Approver (likely from SIG/area to which feature belongs):
  • Feature target (which target equals to which milestone):
    • 1.7
      • node authorizer and noderestriction admission beta release
    • 1.13
    • 1.14
      • restrict node address self-modification
      • stable release
@liggitt

This comment has been minimized.

Member

liggitt commented Apr 26, 2017

@liggitt

This comment has been minimized.

Member

liggitt commented Apr 26, 2017

@idvoretskyi idvoretskyi added this to In Progress in Kubernetes 1.7 features May 3, 2017

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue May 16, 2017

Merge pull request #45775 from liggitt/mirror-pod-validation
Automatic merge from submit-queue (batch tested with PRs 44337, 45775, 45832, 45574, 45758)

Tighten validation of mirror pod annotations

Tightens validation for pods with a mirror pod annotation:
1. spec.nodeName must be set
2. makes the mirror pod annotation immutable
3. starts validating pod-specific annotations during pod status update

None of these changes affect usage of the mirror pod annotation by kubelets, which only set it on pod creation (verified this is true back to 1.5.x)

the second commit updates the pod validation tests to look for specific error messages (best reviewed ignoring whitespace changes)

This is the validation portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/enhancements#279

```release-note
Mirror pods must now indicate the nodeName they are bound to on creation. The mirror pod annotation is now treated as immutable and cannot be added to an existing pod, removed from a pod, or modified.
```

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue May 31, 2017

Merge pull request #46076 from liggitt/node-authorizer
Automatic merge from submit-queue

Node authorizer

This PR implements the authorization portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/enhancements#279:
* Adds a new authorization mode (`Node`) that authorizes requests from nodes based on a graph of related pods,secrets,configmaps,pvcs, and pvs:
  * Watches pods, adds edges (secret -> pod, configmap -> pod, pvc -> pod, pod -> node)
  * Watches pvs, adds edges (secret -> pv, pv -> pvc)
  * When both Node and RBAC authorization modes are enabled, the default RBAC binding that grants the `system:node` role to the `system:nodes` group is not automatically created.
* Tightens the `NodeRestriction` admission plugin to require identifiable nodes for requests from users in the `system:nodes` group.

This authorization mode is intended to be used in combination with the `NodeRestriction` admission plugin, which limits the pods and nodes a node may modify. To enable in combination with RBAC authorization and the NodeRestriction admission plugin:
* start the API server with `--authorization-mode=Node,RBAC --admission-control=...,NodeRestriction,...`
* start kubelets with TLS boostrapping or with client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>`

```release-note
kube-apiserver: a new authorization mode (`--authorization-mode=Node`) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods.
* Nodes must use client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>` in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements)
* When used in combination with the `RBAC` authorization mode (`--authorization-mode=Node,RBAC`), the `system:node` role is no longer automatically granted to the `system:nodes` group.
```

```release-note
RBAC: the automatic binding of the `system:node` role to the `system:nodes` group is deprecated and will not be created in future releases. It is recommended that nodes be authorized using the new `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific ClusterRoleBinding.
```

Follow-up:
- [ ] enable e2e CI environment with admission and authorizer enabled (blocked by kubelet TLS bootstrapping enablement in #40760)
- [ ] optionally enable this authorizer and admission plugin in kubeadm
- [ ] optionally enable this authorizer and admission plugin in kube-up

@liggitt liggitt added stage/beta and removed stage/alpha labels Jun 2, 2017

@liggitt liggitt changed the title from Reduce scope of node access to API server to Limit API access by nodes Jun 3, 2017

@liggitt liggitt changed the title from Limit API access by nodes to Limit node access to API Jun 3, 2017

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue Jun 13, 2017

Merge pull request #46921 from liggitt/kubemark-node-auth
Automatic merge from submit-queue (batch tested with PRs 46441, 43987, 46921, 46823, 47276)

Enable Node authorizer and NodeRestriction admission in kubemark

xref kubernetes/enhancements#279

We want to ensure scale testing covers use of the authorizer/admission pair that partitions nodes. This includes enabling the authorizer, which populates a graph of existing nodes and pods.

Kubemark is still running all nodes with a single credential, so a follow-up step is to generate unique credentials per node (or enable TLS bootstrapping) and remove the temporary rolebinding added in this PR so the node authorizer is the one authorizing each call by a hollow node.

@liggitt liggitt added this to the v1.10 milestone Dec 5, 2017

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented Jan 22, 2018

/kind feature

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented Jan 22, 2018

@liggitt am I right that it's still beta for 1.10?

@liggitt

This comment has been minimized.

Member

liggitt commented Jan 22, 2018

correct, it is beta until kubernetes/community#911 is resolved

work on that is occurring in the 1.10 timeframe

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented Jan 22, 2018

@liggitt great, thanks

@adohe

This comment has been minimized.

Member

adohe commented Jan 31, 2018

@liggitt I am looking at this feature description limit nodes' access to specific APIs, so that they may only modify their own Node API object, only modify Pod objects bound to themselves, and only retrieve secrets and configmaps referenced by pods bound to themselves, IIUC if I enable this feature, the node access to API will be limited to quite narrow scope, and in a multitenant environment, this will bring more high level security.

@Bradamant3

This comment has been minimized.

Member

Bradamant3 commented Mar 2, 2018

@liggitt does this need docs for 1.10? Maybe not? (can't quite tell from comments) Could you please update the 1.10 tracking spreadsheet and any related docs PR? Thanks!

@liggitt

This comment has been minimized.

Member

liggitt commented Mar 2, 2018

@liggitt does this need docs for 1.10? Maybe not? (can't quite tell from comments) Could you please update the 1.10 tracking spreadsheet and any related docs PR? Thanks!

no updates in 1.10

@justaugustus

This comment has been minimized.

Member

justaugustus commented Apr 17, 2018

@mikedanese @liggitt
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

  • Description
  • Milestone
  • Assignee(s)
  • Labels:
    • stage/{alpha,beta,stable}
    • sig/*
    • kind/feature

cc @idvoretskyi

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue May 16, 2018

Merge pull request #63167 from liggitt/taint-modification
Automatic merge from submit-queue (batch tested with PRs 63167, 63357). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Prevent nodes from updating taints

Prevents kubelets from modifying or removing taints on update.

Nodes can set taints when they register themselves, but do not update/remove those taints after creation (that is done by the node controller based on reported node conditions).

xref kubernetes/community#911 kubernetes/enhancements#279

/sig node
/sig auth
/sig scheduling

/assign @mikedanese @k82cn

```release-note
The NodeRestriction admission plugin now prevents kubelets from modifying/removing taints applied to their Node API object.
```
@mistyhacks

This comment has been minimized.

mistyhacks commented May 24, 2018

@liggitt @mikedanese please fill out the appropriate line item of the
1.11 feature tracking spreadsheet
and open a placeholder docs PR against the
release-1.11 branch
by 5/25/2018 (tomorrow as I write this) if new docs or docs changes are
needed and a relevant PR has not yet been opened.

@justaugustus

This comment has been minimized.

Member

justaugustus commented Jun 4, 2018

@liggitt @mikedanese -- We're doing one more sweep of the 1.11 Features tracking spreadsheet.
Would you mind filling in any incomplete / blank fields for this feature's line item?

@kacole2

This comment has been minimized.

Contributor

kacole2 commented Jul 23, 2018

@saad-ali This feature was worked on in the previous milestone, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.12 since you've marked it as beta in 1.12. This still has the 1.11 milestone as well so we need to update it accordingly.

If there are any updates, please explicitly ping @justaugustus, @kacole2, @robertsandoval, @rajendar38 to note that it is ready to be included in the Features Tracking Spreadsheet for Kubernetes 1.12.


Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

  • Docs deadline (open placeholder PRs): 8/21
  • Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

@justaugustus justaugustus removed this from the v1.11 milestone Jul 31, 2018

@tallclair

This comment has been minimized.

Member

tallclair commented Jul 31, 2018

Still beta in 1.12, with some incremental improvements.

@kacole2

This comment has been minimized.

Contributor

kacole2 commented Oct 8, 2018

Hi
This enhancement has been tracked before, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.13. This release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement if there is a high level of confidence it will meet the following deadlines:

  • Docs (open placeholder PRs): 11/8
  • Code Slush: 11/9
  • Code Freeze Begins: 11/15
  • Docs Complete and Reviewed: 11/27

Please take a moment to update the milestones on your original post for future tracking and ping @kacole2 if it needs to be included in the 1.13 Enhancements Tracking Sheet

Thanks!

@liggitt

This comment has been minimized.

Member

liggitt commented Oct 24, 2018

Yes, limiting self-labeling capabilities is being worked on for 1.13

@kacole2

This comment has been minimized.

Contributor

kacole2 commented Oct 24, 2018

@liggitt still considered beta or graduating to stable?

@liggitt

This comment has been minimized.

Member

liggitt commented Oct 24, 2018

Updated milestones to reflect the work targeted for 1.13. Node address reporting remains before this will be promoted to stable (looking toward 1.14 for that)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment