Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

harden the default RBAC discovery clusterrolebindings #73807

Merged
merged 1 commit into from
Mar 2, 2019

Conversation

dekkagaijin
Copy link
Contributor

@dekkagaijin dekkagaijin commented Feb 7, 2019

Implements kubernetes/enhancements#789

/kind feature

Default RBAC policy no longer grants access to discovery and permission-checking APIs (used by `kubectl auth can-i`) to *unauthenticated* users. Upgraded clusters preserve prior behavior, but cluster administrators wishing to grant unauthenticated users access in new clusters will need to explicitly opt-in to expose the discovery and/or permission-checking APIs:
* `kubectl create clusterrolebinding anonymous-discovery --clusterrole=system:discovery --group=system:unauthenticated`
* `kubectl create clusterrolebinding anonymous-access-review --clusterrole=system:basic-user --group=system:unauthenticated`

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Feb 7, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @dekkagaijin. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 7, 2019
@k8s-ci-robot k8s-ci-robot added sig/auth Categorizes an issue or PR as relevant to SIG Auth. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Feb 7, 2019
@dekkagaijin
Copy link
Contributor Author

/assign @liggitt

@liggitt
Copy link
Member

liggitt commented Feb 7, 2019

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 7, 2019
@dekkagaijin dekkagaijin changed the title [WIP] harden the default RBAC discovery clusterrolebindings harden the default RBAC discovery clusterrolebindings Feb 8, 2019
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 8, 2019
@dekkagaijin
Copy link
Contributor Author

@liggitt OK to review what I have now, don't really know the most sane way to test the storage_rbac change. I'll flesh out the release note to include the possible pitfalls and the sample kubectl escape hatches

pkg/registry/rbac/rest/storage_rbac.go Outdated Show resolved Hide resolved
const (
PostStartHookName = "rbac/bootstrap-roles"
discoveryClusterRoleName = "system:discovery"
publicInfoViewerClusterRoleName = "system:public-info-viewer"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of hard-coding role names here, get a set of map[string]ClusterRoleBinding (source name -> destination clusterrolebinding template) from the bootstrap policy data (e.g. SplitClusterRoleBindings()), then iterate over them in primeSplitClusterRoleBindings

inside that method, make a DeepCopy() of the destination clusterrolebinding template and actually copy over the labels/annotations/subjects from the source clusterrolebinding if present

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still outstanding

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@liggitt
Copy link
Member

liggitt commented Feb 23, 2019

don't really know the most sane way to test the storage_rbac change

@jennybuckley has a great example of an integration test that:

  1. starts a shared etcd
  2. starts an apiserver and lets it come up
  3. monkeys with stuff
  4. shuts down the apiserver and starts a new one against the same etcd data
  5. verifies stuff

that sounds like exactly what you want

@dekkagaijin
Copy link
Contributor Author

@liggitt thanks, I'll finish this up tomorrow (crosses fingers) after perf/promo is done

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Feb 27, 2019
test/integration/auth/rbac_test.go Outdated Show resolved Hide resolved
test/integration/auth/rbac_test.go Outdated Show resolved Hide resolved
@liggitt
Copy link
Member

liggitt commented Feb 28, 2019

a couple comments on the test, and one outstanding comment on where the old binding -> new binding data should live. lgtm otherwise

@dekkagaijin
Copy link
Contributor Author

done, PTAL @liggitt

@dekkagaijin dekkagaijin force-pushed the discovery-hardening branch 2 times, most recently from 88ef13a to f1e6e9e Compare March 1, 2019 06:03
@dekkagaijin
Copy link
Contributor Author

/test pull-kubernetes-verify

@liggitt
Copy link
Member

liggitt commented Mar 1, 2019

an attempt was made to switch to go1.12 last night... govet failure is spurious, retest should resolve it

@dekkagaijin
Copy link
Contributor Author

@liggitt that was the trick, PTAL :)

// If source ClusterRoleBinding does not exist, do nothing.
existingRoleBinding, err := clusterRoleBindingClient.ClusterRoleBindings().Get(existingBindingName, metav1.GetOptions{})
if apierrors.IsNotFound(err) {
return nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the return nil lines inside the loop should be continue statements

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

@liggitt
Copy link
Member

liggitt commented Mar 2, 2019

/lgtm
/approve
/milestone v1.14

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 2, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.14 milestone Mar 2, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dekkagaijin, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 2, 2019
@k8s-ci-robot k8s-ci-robot merged commit f160356 into kubernetes:master Mar 2, 2019
@dekkagaijin dekkagaijin deleted the discovery-hardening branch March 2, 2019 05:50
@redbaron
Copy link
Contributor

redbaron commented Mar 3, 2019

Release notes say "Upgraded clusters preserve prior behavior", how can administrators of upgraded clusters move to hardened RBAC?

@dekkagaijin
Copy link
Contributor Author

@redbaron (nice username btw) one only need remove the system:authenticated Subject group form the system:discovery and system:basic-user ClusterRoleBindings.

I don't think that there's a universal, one- or two-line command to trim the old Bindings since kubectl patch isn't overly smart about array elements, but if a general description of the change is fine then I can add the previous sentence to the release notes.

@liggitt
Copy link
Member

liggitt commented Mar 5, 2019

I don't think that there's a universal, one- or two-line command to trim the old Bindings since kubectl patch isn't overly smart about array elements

kubectl auth reconcile --remove-extra-subjects -f ... was made for this, but needs an input file containing only the trimmed clusterrolebindings

@dekkagaijin
Copy link
Contributor Author

@redbaron Here's something I came up with to harden upgraded clusters to the new RBAC discovery roles:

cat <<EOF | kubectl auth reconcile --remove-extra-subjects -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: system:basic-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:basic-user
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:authenticated
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: system:discovery
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:discovery
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:authenticated
EOF

k8s-ci-robot pushed a commit to kubernetes/website that referenced this pull request Mar 12, 2019
)

* Document changes to default RBAC discovery ClusterRole(Binding)s

Documentation for kubernetes/enhancements#789 and kubernetes/kubernetes#73807

* documentation review feedback
k8s-ci-robot pushed a commit to kubernetes/website that referenced this pull request Mar 25, 2019
* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Document timeout attribute for kms-plugin. (#12158)

See 72540.

* Official documentation on Poseidon/Firmament, a new multi-scheduler  (#12343)

* Removed the old version of the Poseidon documentation. Incorrect location.

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S (#12069)

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Updated the document as per review comments. Added config details.

* Updated the document as per the latest review comments. Fixed nits

* Made changes as per latest suggestions.

* Some more changes added.

* Updated as per suggestions.

* Changed the release process section.

* SIG Docs edits

Small edits to match style guidelines.

* add plus to feature state

* capitalization

* revert feature state shortcode

since this is a Kubernetes extension, not a direct feature, it shouldn't use the regular feature state tagging.

(cherry picked from commit 7730c15)

* Remove initializers from doc. It will be removed in 1.14 (#12331)

* kubeadm: Document CRI auto detection functionality (#12462)

Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>

* Minor doc change for GAing Pod DNS Config (#12514)

* Graduate ExpandInUsePersistentVolumes feature to beta (#10574)

* Rename 2018-11-07-grpc-load-balancing-with-linkerd.md.md file (#12594)

* Add dynamic percentage of node scoring to user docs (#12235)

* Add dynamic percentage of node scoring to user docs

* addressed review comments

* delete special symbol (#12445)

* Update documentation for VolumeSubpathEnvExpansion (#11843)

* Update documentation for VolumeSubpathEnvExpansion

* Address comments - improve descriptions

* Graduate Pod Priority and Preemption to GA (#12428)

* Added Instana links to the documentation (#12977)

* Added link to the Instana Kubernetes integration

* Added Instana link for services section

Added Instana and a link to the Kubernetes integration to the analytics services section and broadened the scope to APM, monitoring and analytics.

* Oxford comma /flex

* More Oxford commas, because they matter

* Update kubectl plugins to stable (#12847)

* documentation for CSI topology beta (#12889)

* Document changes to default RBAC discovery ClusterRole(Binding)s (#12888)

* Document changes to default RBAC discovery ClusterRole(Binding)s

Documentation for kubernetes/enhancements#789 and kubernetes/kubernetes#73807

* documentation review feedback

* CSI raw block to beta (#12931)

* Change incorrect string raw to block (#12926)

Fixes #12925

* Update documentation on node OS/arch labels (#12976)

These labels have been promoted to GA:
kubernetes/enhancements#793

* local pv GA doc updates (#12915)

* Publish CRD OpenAPI Documentation (#12910)

* add documentation for CustomResourcePublishOpenAPI

* address comments

fix links, ordered lists, style and typo

* kubeadm: add document for upgrading from 1.13 to 1.14 (single CP and HA) (#13189)

* kubeadm: add document for upgrading from 1.13 to 1.14

- remove doc for upgrading 1.10 -> 1.11

* kubeadm: apply amends to upgrade-1.14 doc

* kubeadm: apply amends to upgrade-1.14 doc (part2)

* kubeadm: apply amends to upgrade-1.14 doc (part3)

* kubeadm: add note about "upgrade node experimental-control-plane"

+ add comment about `upgrade plan`

* kubeadm: add missing "You should see output similar to this"

* fix bullet indentation (#13214)

* mark PodReadinessGate GA (#12800)

* Update RuntimeClass documentation for beta (#13043)

* Update RuntimeClass documentation for beta

* Update feature gate & add upgrade section

* formatting fixes

* Highlight upgrade action required

* Address feedback

* CSI ephemeral volume alpha documentation (#10934)

* update kubectl documentation (#12867)

* update kubectl documentation

* add document for Secret/ConfigMap generators

* replace `kubectl create -f` by `kubectl apply -f`

* Add page for kustomization support in kubectl

* fix spelling errors and address comments

* Documentation for Windows GMSA feature (#12936)

* Documentation for Windows GMSA feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Enhancements to GMSA docs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix GMSA link

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add GMSA feature flag in feature flag list

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Relocate GMSA to container configuration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add example for container spec

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Remove changes in Windows index

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Rearrange the steps into two sections and other edits

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add reference to script to generate GMSA YAMLs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Some more clarifications for GMSA

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* HugePages graduated to GA (#13004)

* HugePages graduated to GA

* fixing nit for build

* Docs for node PID limiting (kubernetes/kubernetes#73651) (#12932)

* kubeadm: update the reference documentation for 1.14 (#12911)

* kubeadm: update list of generated files for 1.14

NOTE: PLACEHOLDERS! these files are generated by SIG Docs each
release, but we need them to pass the k/website PR CI.

- add join_phase* (new sub phases of join)
- add init_phase_upload-certs.md (new upload certs phase for init)
- remove alpha-preflight (now both init and join have this)

* kubeadm: update reference docs includes for 1.14

- remove includes from alpha.md
- add upload-certs to init-phase.md
- add join-phase.md and it's phases

* kubeadm: update the editorial content of join and init

- cleanup master->control-plane node
- add some notes about phases and join
- remove table about pre-pulling images
- remove outdated info about self-hosting

* kubeadm: update target release for v1alpha3 removal

1.14 -> 1.15

* kubeadm: copy edits for 1.14 reference docs (part1)

* kubeadm: use "shell" for code blocks

* kubeadm: update the 1.14 HA guide (#13191)

* kubeadm: update the 1.14 HA guide

* kubeadm: try to fix note/caution indent in HA page

* kubeadm: fix missing sudo and minor amends in HA doc

* kubeadm: apply latest amends to the HA doc for 1.14

* fixed a few missed merge conflicts

* Admission Webhook new features doc (#12938)

- kubernetes/kubernetes#74998
- kubernetes/kubernetes#74477
- kubernetes/kubernetes#74562

* Clarifications and fixes in GMSA doc (#13226)

* Clarifications and fixes in GMSA doc

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to fix bullets

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reword application of sample gmsa

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* RunAsGroup documentation for Progressing this to Beta (#12297)

* start serverside-apply documentation (#13077)

* start serverside-apply documentation

* add more concept info on server side apply

* Update api concepts

* Update api-concepts.md

* fix style issues

* Document CSI update (#12928)

* Document CSI update

* Finish CSI documentation

Also fix mistake with  ExpandInUsePersistentVolumes documented as beta

* Overall docs for CSI Migration feature (#12935)

* Placeholder docs for CSI Migration feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address CR comments and update feature gates

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add mappings for CSI plugins

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add sections for AWS and GCE PD migration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add docs for Cinder and CSI Migration info

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Clarify scope to volumes with file system

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Change the format of EBS and Cinder CSI Migration sections to follow the GCE template

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Windows documentation updates for 1.14 (#12929)

* Updated the note to indicate doc work for 1.14

* first attempt at md export from gdoc

* simplifyig

* big attempt

* moving DRAFT windows content to PR for review

* moving content to PR in markdown for review

* updated note tags

* Delete windows-contributing.md

deleting this file as it is already ported to the github contributor guide

* fixed formatting in intro and cluster setup guide

* updating formatting for running containers guide

* rejiggered end of troubleshooting

* fixed minor typos

* Clarified the windows binary download step

* Update _index.md

making updates based on feedback

* Update _index.md

updating ovn-kubernetes docs

* Update _index.md

* Update _index.md

* updating relative docs links

updating all the links to be relative links to /docs

* Update _index.md

* Update _index.md

updates for windows services and ovn-kubernetes

* formatted for correct step numbering

* fix typos

* Update _index.md

updates for flannel PR in troubleshooting

* Update _index.md

* Update _index.md

updating a few sections like roadmap, services, troubleshooting/filing tickets

* Update _index.md

* Update _index.md

* Update _index.md

* Fixed a few whitespace issues

* Update _index.md

* Update _index.md

* Update _index.md

* add section on upgrading CoreDNS (#12909)

* documentation for kubelet resource metrics endpoint (#12934)

* windows docs updates for 1.14 (#13279)

* Delete sample-l2bridge-wincni-config.json

this file is not used anywhere

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Rename content/en/docs/getting-started-guides/windows/_index.md to content/en/docs/setup/windows/_index.md

moving to new location

* Delete flannel-master-kubectl-get-ds.png

* Delete flannel-master-kubeclt-get-pods.png

* Delete windows-docker-error.png

* Add files via upload

* Rename _index.md to add-windows-nodes.md

* Create _index.md

* Update _index.md

* Update add-windows-nodes.md

* Update add-windows-nodes.md

* Create user-guide-windows-nodes.md

* Create user-guide-windows-containers.md

* Update and rename add-windows-nodes.md to intro-windows-nodes.md

* Update user-guide-windows-containers.md

* Rename intro-windows-nodes.md to intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update _index.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

fixing the pause image

* Update intro-windows-in-kubernetes.md

changing tables from html to MD

* Update user-guide-windows-nodes.md

converting tables from HTML to MD

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

updating the numbering , even though it messes up the notes a little bit. Jim will file a ticket to follow up

* Update user-guide-windows-nodes.md

* update to windows docs for 1.14 (#13322)

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update intro-windows-in-kubernetes.md (#13344)

* server side apply followup (#13321)

* change some parts of serverside apply docs in response to comments

* fix typos and wording

* Update config.toml (#13365)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants