Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable webhook authn in test #82551

Open
wants to merge 3 commits into
base: master
from

Conversation

@tallclair
Copy link
Member

commented Sep 10, 2019

What type of PR is this?
/kind cleanup

What this PR does / why we need it:

Enable the Node authentication & restriction E2E tests in most test suites. Due to the way ginkgo skip/focus works, there isn't a good way of including feature tests in default test suites. So, this PR introduces a new type of test tag: [Config:...] for tests that should be included in the default suite but depend on a particular configuration.

Enable webhook authentication for nodes in the GCE test environment. This lets us enable the Node Authentication tests in E2E.

Does this PR introduce a user-facing change?:

NONE

/sig auth
/sig node
/sig testing
/priority important-soon
/assign @mikedanese

@tallclair

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2019

/hold

I'm just going to enable the tests is this PR.

Enable NodeAuthorizer & NodeAuthenticator tests by default
Introduce a new [Config:...] tag scheme for tests that should be enabled by
deafult, but depend on a specific configuration.

@k8s-ci-robot k8s-ci-robot added size/S and removed size/XS labels Sep 10, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2019

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tallclair
To complete the pull request process, please assign mikedanese
You can assign the PR to them by writing /assign @mikedanese in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tallclair

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2019

/cc @liggitt

@k8s-ci-robot k8s-ci-robot requested a review from liggitt Sep 10, 2019

@tallclair

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2019

@k8s-ci-robot k8s-ci-robot requested a review from BenTheElder Sep 10, 2019

@tallclair

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2019

/hold cancel

@@ -54,7 +54,7 @@ var _ = SIGDescribe("[Feature:NodeAuthenticator]", func() {
gomega.Expect(len(sa.Secrets)).NotTo(gomega.BeZero())
})

ginkgo.It("The kubelet's main port 10250 should reject requests with no credentials", func() {
ginkgo.It("The kubelet's main port 10250 should reject requests with no credentials [Config:Kubelet:NoAnonymousAuth]", func() {

This comment has been minimized.

Copy link
@BenTheElder

BenTheElder Sep 11, 2019

Member

this sounds like a feature of the cluster. we now need to update other test suite configs to skip [Config:.*] ?

This comment has been minimized.

Copy link
@neolit123

neolit123 Sep 11, 2019

Member

yes, while Feature: is a common pattern Config: is AFAIK brand new?

This comment has been minimized.

Copy link
@alejandrox1

alejandrox1 Sep 11, 2019

Contributor

just checked, it is brand new.

Enable the Node authentication & restriction E2E tests in most test suites

This would indeed add the test to most test suites.

This comment has been minimized.

Copy link
@alejandrox1

alejandrox1 Sep 11, 2019

Contributor

If pull-kubernetes-e2e-gce it would be safe (fingers crossed) to merge this, no?

This comment has been minimized.

Copy link
@BenTheElder

BenTheElder Sep 11, 2019

Member

If pull-kubernetes-e2e-gce it would be safe (fingers crossed) to merge this, no?

it may break CI on other deployment tools like kops etc..

This comment has been minimized.

Copy link
@tallclair

tallclair Sep 11, 2019

Author Member

To clarify, I specifically don't want test suites to skip [Config:.*], which is why I didn't just keep it as a feature. Essentially, I want this to be opt-out rather than opt-in.

I think the real problem is that we don't have a good way of accessing the the cluster configuration in the tests. What I really want to do is say: "if node webhook authentication is disabled, skip this test". Using [Feature:] tags doesn't really scale, as we can see from a number of tests that were added but never run.

#33706 was another attempt at addressing this.

I also want to make sure the configuration is explicit in the test configuration, rather than scraped from the cluster. If we scraped the configuration from the cluster, we wouldn't catch bugs where a feature was accidentally disabled.

This comment has been minimized.

Copy link
@neolit123

neolit123 Sep 11, 2019

Member

not having a good way to determine how the cluster is configured is definitely a problem.
/assign @timothysc @spiffxp

This comment has been minimized.

Copy link
@timothysc

timothysc Sep 11, 2019

Member

I think the real problem is that we don't have a good way of accessing the the cluster configuration in the tests. What I really want to do is say: "if node webhook authentication is disabled, skip this test". Using [Feature:] tags doesn't really scale, as we can see from a number of tests that were added but never run.

This is what component config working group is doing, but currently [tags] are 'kindof' reserved.

@k8s-ci-robot k8s-ci-robot requested a review from alejandrox1 Sep 11, 2019

@alejandrox1
Copy link
Contributor

left a comment

/milestone v1.17

@k8s-ci-robot k8s-ci-robot added this to the v1.17 milestone Sep 11, 2019

@@ -534,3 +534,6 @@ GCE_PRIVATE_CLUSTER="${KUBE_GCE_PRIVATE_CLUSTER:-false}"
ETCD_LISTEN_CLIENT_IP=0.0.0.0

GCE_UPLOAD_KUBCONFIG_TO_MASTER_METADATA=true

# Enable token webhook authentication in the Kubelet
KUBELET_TEST_ARGS="${KUBELET_TEST_ARGS:-} --authentication-token-webhook=true"

This comment has been minimized.

Copy link
@timothysc

timothysc Sep 11, 2019

Member

I would ask that you broadly enable or change the default then.

This comment has been minimized.

Copy link
@tallclair

tallclair Sep 11, 2019

Author Member

I don't think we want to enable token webhook authentication by default. There are still some issues with it, and reasons to keep it off (lack of audiences support comes to mind).

I think it makes sense to enable non-default features in the testing environment to test those features.

@@ -54,7 +54,7 @@ var _ = SIGDescribe("[Feature:NodeAuthenticator]", func() {
gomega.Expect(len(sa.Secrets)).NotTo(gomega.BeZero())
})

ginkgo.It("The kubelet's main port 10250 should reject requests with no credentials", func() {
ginkgo.It("The kubelet's main port 10250 should reject requests with no credentials [Config:Kubelet:NoAnonymousAuth]", func() {

This comment has been minimized.

Copy link
@timothysc

timothysc Sep 11, 2019

Member

I think the real problem is that we don't have a good way of accessing the the cluster configuration in the tests. What I really want to do is say: "if node webhook authentication is disabled, skip this test". Using [Feature:] tags doesn't really scale, as we can see from a number of tests that were added but never run.

This is what component config working group is doing, but currently [tags] are 'kindof' reserved.

@k8s-ci-robot k8s-ci-robot added size/M and removed size/S labels Sep 11, 2019

@tallclair

This comment has been minimized.

Copy link
Member Author

commented Sep 11, 2019

Fixed the Node Authentication tests to account for kubelet not listening on the external interface.

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Sep 11, 2019

@tallclair: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-verify 9a019c2 link /test pull-kubernetes-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.