New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set default enabled admission plugins by official document #58684

Merged
merged 1 commit into from Feb 22, 2018

Conversation

@hzxuzhonghu
Member

hzxuzhonghu commented Jan 23, 2018

What this PR does / why we need it:

https://kubernetes.io/docs/admin/admission-controllers/#is-there-a-recommended-set-of-admission-controllers-to-use

recommend running the following set of admission controllers

If you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard).  See [https://kubernetes.io/docs/admin/admission-controllers/] for explanation of admission control.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Set default enabled admission plugins `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`
@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Jan 23, 2018

/assign @sttts @deads2k

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Jan 23, 2018

And after this , the document also need revise.

@k8s-ci-robot k8s-ci-robot added size/S and removed size/M labels Jan 23, 2018

defaultOffPlugins := sets.NewString(AllOrderedPlugins...)
defaultOffPlugins.Delete(lifecycle.PluginName)
defaultOnPlugins := sets.NewString(
lifecycle.PluginName,

This comment has been minimized.

@deads2k

deads2k Jan 23, 2018

Contributor

Alias the packages to admission plugin names so we can recognize these. I think this is namespacelifecycle.

This comment has been minimized.

@hzxuzhonghu

hzxuzhonghu Jan 24, 2018

Member

Add later.

@@ -79,7 +79,7 @@ func NewAdmissionOptions() *AdmissionOptions {
// after all the mutating ones, so their relative order in this list
// doesn't matter.
RecommendedPluginOrder: []string{lifecycle.PluginName, initialization.PluginName, mutatingwebhook.PluginName, validatingwebhook.PluginName},
DefaultOffPlugins: sets.NewString(initialization.PluginName, mutatingwebhook.PluginName, validatingwebhook.PluginName),

This comment has been minimized.

@deads2k

deads2k Jan 23, 2018

Contributor

split this out, it is non-contentious

@deads2k

This comment has been minimized.

Contributor

deads2k commented Jan 23, 2018

This is a huge change. I'm strongly in favor, but we need to agree that we're willing to do this.

  1. People who previously controlled the admission chain using --admission-control will be fine and their options are still respected.
  2. People who specified no admission chains had a scarily exposed cluster they could not run securely, but now they'll suddenly get the admission chain and potentially breaks as things like service accounts will now be require. We've long recommended this, but we've never turned them on by default before.

@smarterclayton @bgrant0607 @kubernetes/api-approvers @lavalamp @erictune @liggitt for comment on having a default admission chain.

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Jan 25, 2018

ping @smarterclayton @bgrant0607 @kubernetes/api-approvers @lavalamp @erictune @liggitt for comment on having a default admission chain.

@lavalamp

This comment has been minimized.

Member

lavalamp commented Jan 30, 2018

I am in favor of fixing this but I'm not sure this is the right way to do it.

  • I think the UX is super confusing. It's not good to break naive users (e.g., I bet this breaks the local-up script). So, at a minimum I think we need users to opt-in to the new system.
  • Changing entries in a default-on list is always going to be a breaking change. I think the right way to do this is to have apiserver store the list, probably somewhere in etcd. If the list is unset, then we write the defaults; otherwise, we use the list. That way, we could change the defaults without totally breaking everyone on upgrades.
@deads2k

This comment has been minimized.

Contributor

deads2k commented Jan 30, 2018

I think the UX is super confusing. It's not good to break naive users (e.g., I bet this breaks the local-up script). So, at a minimum I think we need users to opt-in to the new system.

I'm ok with having a --disable-admission flag (which is what this would be). I'd like to have a timeline for having --disable-admission default to false.

Changing entries in a default-on list is always going to be a breaking change. I think the right way to do this is to have apiserver store the list, probably somewhere in etcd. If the list is unset, then we write the defaults; otherwise, we use the list. That way, we could change the defaults without totally breaking everyone on upgrades.

The idea of a "default on" list is that we think these are a good default for a safe and sane cluster, it's not a guarantee of particular plugins being on or off. Think about things like the PodToleration plugin. If you don't enable it in the same release that you added taint and toleration support, the feature presents
a risk to your cluster. Most cluster-admins probably want to have their safe cluster continue being safe.

@sttts

This comment has been minimized.

Contributor

sttts commented Jan 30, 2018

How many (non-toy/dev) clusters are out there without --admission-control?

Compared with the big switch to RBAC-by-default, any serious cluster probably has admission plugins enabled anyway. This was very different for RBAC.

@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Feb 4, 2018

We need to think carefully about what we're trying to achieve.

If we want to make it easier for users to stay up to date with recommended admission controllers, I suggest a new flag along the lines of --enable-recommended-admission-controllers, and a warning if the existing flag is otherwise left unset.

@erictune

This comment has been minimized.

Member

erictune commented Feb 6, 2018

I'm in favor of making this change, and having it be opt-out.

@erictune

This comment has been minimized.

Member

erictune commented Feb 6, 2018

The release note needs to be more specific. Maybe append something like this:

If you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard).  See [https://kubernetes.io/docs/admin/admission-controllers/] for explanation of admission control.
@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 6, 2018

@erictune ok, will update today.

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 6, 2018

so many case failed with ServiceAccount plugin enabled.

@deads2k

This comment has been minimized.

Contributor

deads2k commented Feb 6, 2018

so many case failed with ServiceAccount plugin enabled.

controllers not running maybe?

@k8s-ci-robot k8s-ci-robot added size/M and removed size/S labels Feb 9, 2018

@k8s-ci-robot k8s-ci-robot added size/S and removed size/M labels Feb 9, 2018

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 9, 2018

/test pull-kubernetes-bazel-test

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 11, 2018

rebased

@deads2k

This comment has been minimized.

Contributor

deads2k commented Feb 21, 2018

Sorry, I didn't see that this got itself sorted out.

/lgtm

squash up when you rebase

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 22, 2018

@deads2k @sttts squashed and rebased

@deads2k

This comment has been minimized.

Contributor

deads2k commented Feb 22, 2018

/lgtm

@k8s-merge-robot k8s-merge-robot removed the lgtm label Feb 22, 2018

@hzxuzhonghu

This comment has been minimized.

Member

hzxuzhonghu commented Feb 22, 2018

run gofmt

@deads2k

This comment has been minimized.

Contributor

deads2k commented Feb 22, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Feb 22, 2018

@deads2k deads2k added this to the v1.10 milestone Feb 22, 2018

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Feb 22, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Feb 22, 2018

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Feb 22, 2018

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-merge-robot k8s-merge-robot merged commit 270148d into kubernetes:master Feb 22, 2018

12 of 13 checks passed

Submit Queue Required Github CI test is not green: pull-kubernetes-verify
Details
cla/linuxfoundation hzxuzhonghu authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce Job succeeded.
Details
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-unit Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

@hzxuzhonghu hzxuzhonghu deleted the hzxuzhonghu:default-enabled-admission branch Feb 22, 2018

setdefault.PluginName, //DefaultStorageClass
defaulttolerationseconds.PluginName, //DefaultTolerationSeconds
mutatingwebhook.PluginName, //MutatingAdmissionWebhook
validatingwebhook.PluginName, //ValidatingAdmissionWebhook

This comment has been minimized.

@lavalamp

lavalamp May 23, 2018

Member

Why are webhooks off by default?

This comment has been minimized.

@liggitt

liggitt May 23, 2018

Member

this looks to make them on by default, not off

This comment has been minimized.

@lavalamp

lavalamp May 23, 2018

Member

My bad. I was reading the function name, not the variable name. Sorry for the noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment