Extend node e2e test suite with containerized Kubelet #56250

ingvagabund · 2017-11-22T21:01:22Z

How to run it?

make test-e2e-node \
    TEST_ARGS='--kubelet-containerized=true --hyperkube-image=hyperkube-amd64:1.9 --kubelet-flags="<FLAGS>"' \
    FOCUS="Conformance"

ingvagabund · 2017-11-22T21:20:33Z

How relevant is it to implement the non-systemd variant?

ingvagabund · 2017-11-23T12:15:11Z

/test pull-kubernetes-unit

sjenning · 2017-11-27T17:15:30Z

test/e2e_node/services/kubelet.go

+				"-v", "/etc/machine-id:/etc/machine-id:ro",
+				"-v", "/etc/systemd/system:/host-etc/systemd/system",
+				"-v", filepath.Dir(kubeconfigPath)+":/etc/kubernetes",
+				"-v", "/lib/modules:/lib/modules",


loading kernel modules?

I got inspired by the openshift-node service file. If not needed, I can remove it.

I think we can remove these two (at least).

sjenning · 2017-11-27T17:16:44Z

test/e2e_node/services/kubelet.go

+				"-e HOST=/rootfs", "-e HOST_ETC=/host-etc",
+				"-v", "/etc/localtime:/etc/localtime:ro",
+				"-v", "/etc/machine-id:/etc/machine-id:ro",
+				"-v", "/etc/systemd/system:/host-etc/systemd/system",


what is this needed for?

I got inspired by the openshift-node service file. If not needed, I can remove it.

sjenning · 2017-11-27T17:17:26Z

How relevant is it to implement the non-systemd variant?

What specifically makes this systemd only?

ingvagabund · 2017-11-27T21:56:00Z

What specifically makes this systemd only?

If the systemd-run binary is present, the systemd service is created. Otherwise, a binary on a background is run. So it is needed by the non-systemd OSes.

sjenning · 2017-11-27T23:16:37Z

If the systemd-run binary is present, the systemd service is created. Otherwise, a binary on a background is run. So it is needed by the non-systemd OSes.

Ah I see. So you are just adding support for the systemd path in this PR atm.

How much trouble would it be to implement the non-systemd path as well? This might help our case for upstream containerized kubelet testing.

sjenning · 2017-11-28T16:07:00Z

/sig node
/assign @dchen1107

dims · 2017-11-29T11:15:23Z

/release-note-none

ingvagabund · 2017-11-30T17:10:11Z

@dchen1107 PTAL

dims · 2017-12-01T11:44:43Z

/unassign @dims

derekwaynecarr · 2017-12-04T16:10:17Z

this looks fine to me.

/lgtm
/approve

derekwaynecarr · 2017-12-04T16:11:23Z

/approve no-issue

k8s-github-robot · 2017-12-04T16:12:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, ingvagabund

Associated issue requirement bypassed by: derekwaynecarr

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~test/e2e_node/OWNERS~~ [derekwaynecarr]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

sjenning · 2017-12-04T17:50:42Z

/retest

fejta-bot · 2017-12-09T13:48:48Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

ScorpioCPH

lgtm with minor nit comments :)

ScorpioCPH · 2017-12-12T12:01:00Z

test/e2e_node/services/kubelet.go

@@ -93,6 +97,10 @@ const (
 // startKubelet starts the Kubelet in a separate process or returns an error
 // if the Kubelet fails to start.
 func (e *E2EServices) startKubelet() (*server, error) {
+	if kubeletContainerized && hyperkubeImage == "" {
+		return nil, fmt.Errorf("the --hyperkube-image option must be set")


nit: the --hyperkube-image option must be set if we run kubelet in a docker container

The error message pops up only when the --containerized flag is set. Later, the --hyperkube-image can point to a container of non-docker container runtime. There is always a space for ambiguity. However, the hyperkube + image connection is currently strongly voting for containerized run, so the confusion will be minimal. But yeah, updating the message would help to make the statement clearer :).

k8s-github-robot · 2017-12-16T14:40:10Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-ci-robot · 2017-12-16T15:08:05Z

@ingvagabund: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-unit	`607e863`	link	`/test pull-kubernetes-unit`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-github-robot · 2017-12-16T15:46:37Z

Automatic merge from submit-queue (batch tested with PRs 56250, 56809, 56812, 56792, 56724). If you want to cherry-pick this change to another branch, please follow the instructions here.

…ntal-on-containerized-kubelet Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Containerized kubelet is no longer experimental Blocked by #56250 Given the node e2e Conformance tests over containerized ``Kubelet`` are already running and are published at https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-containerized-conformance-aws-e2e-rhel, we can remove all mentions of the "experimental" keyword. Are there any other place where the "containerized Kubelet" is mentioned as experimental? ```release-note NONE ```

ingvagabund added area/test sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Nov 22, 2017

ingvagabund requested review from sjenning and derekwaynecarr November 22, 2017 21:01

k8s-github-robot assigned dims and derekwaynecarr Nov 22, 2017

ingvagabund force-pushed the extend-node-e2e-test-suite-with-containerized-kubelet branch from 5bb16e6 to 1636395 Compare November 22, 2017 21:19

ingvagabund mentioned this pull request Nov 24, 2017

Introduce kube node e2e tests openshift-eng/aos-cd-jobs#875

Merged

sjenning reviewed Nov 27, 2017

View reviewed changes

ingvagabund changed the title ~~WIP: extend node e2e test suite with containerized Kubelet~~ Extend node e2e test suite with containerized Kubelet Nov 27, 2017

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 27, 2017

WIP: extend node e2e test suite with containerized Kubelet

607e863

ingvagabund force-pushed the extend-node-e2e-test-suite-with-containerized-kubelet branch from 3554f6f to 607e863 Compare November 28, 2017 14:31

ingvagabund mentioned this pull request Nov 28, 2017

Extend the TestGrid with (containerized) node e2e over RHEL on AWS kubernetes/test-infra#5693

Merged

k8s-ci-robot assigned dchen1107 Nov 28, 2017

k8s-ci-robot added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Nov 28, 2017

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Nov 29, 2017

k8s-ci-robot unassigned dims Dec 1, 2017

derekwaynecarr added this to the v1.10 milestone Dec 4, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 4, 2017

k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 4, 2017

ingvagabund mentioned this pull request Dec 11, 2017

Containerized kubelet is no longer experimental #57034

Merged

ScorpioCPH reviewed Dec 12, 2017

View reviewed changes

k8s-github-robot merged commit 0a940bf into kubernetes:master Dec 16, 2017

ingvagabund deleted the extend-node-e2e-test-suite-with-containerized-kubelet branch December 19, 2017 10:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend node e2e test suite with containerized Kubelet #56250

Extend node e2e test suite with containerized Kubelet #56250

ingvagabund commented Nov 22, 2017 •

edited

Loading

ingvagabund commented Nov 22, 2017

ingvagabund commented Nov 23, 2017

sjenning Nov 27, 2017

ingvagabund Nov 27, 2017

sjenning Nov 27, 2017

sjenning Nov 27, 2017

ingvagabund Nov 27, 2017

sjenning commented Nov 27, 2017

ingvagabund commented Nov 27, 2017

sjenning commented Nov 27, 2017

sjenning commented Nov 28, 2017

dims commented Nov 29, 2017

ingvagabund commented Nov 30, 2017

dims commented Dec 1, 2017

derekwaynecarr commented Dec 4, 2017

derekwaynecarr commented Dec 4, 2017

k8s-github-robot commented Dec 4, 2017

sjenning commented Dec 4, 2017

fejta-bot commented Dec 9, 2017

ScorpioCPH left a comment

ScorpioCPH Dec 12, 2017

ingvagabund Dec 15, 2017 •

edited

Loading

ScorpioCPH Dec 15, 2017

k8s-github-robot commented Dec 16, 2017

k8s-ci-robot commented Dec 16, 2017 •

edited

Loading

k8s-github-robot commented Dec 16, 2017

Extend node e2e test suite with containerized Kubelet #56250

Extend node e2e test suite with containerized Kubelet #56250

Conversation

ingvagabund commented Nov 22, 2017 • edited Loading

ingvagabund commented Nov 22, 2017

ingvagabund commented Nov 23, 2017

sjenning Nov 27, 2017

Choose a reason for hiding this comment

ingvagabund Nov 27, 2017

Choose a reason for hiding this comment

sjenning Nov 27, 2017

Choose a reason for hiding this comment

sjenning Nov 27, 2017

Choose a reason for hiding this comment

ingvagabund Nov 27, 2017

Choose a reason for hiding this comment

sjenning commented Nov 27, 2017

ingvagabund commented Nov 27, 2017

sjenning commented Nov 27, 2017

sjenning commented Nov 28, 2017

dims commented Nov 29, 2017

ingvagabund commented Nov 30, 2017

dims commented Dec 1, 2017

derekwaynecarr commented Dec 4, 2017

derekwaynecarr commented Dec 4, 2017

k8s-github-robot commented Dec 4, 2017

sjenning commented Dec 4, 2017

fejta-bot commented Dec 9, 2017

ScorpioCPH left a comment

Choose a reason for hiding this comment

ScorpioCPH Dec 12, 2017

Choose a reason for hiding this comment

ingvagabund Dec 15, 2017 • edited Loading

Choose a reason for hiding this comment

ScorpioCPH Dec 15, 2017

Choose a reason for hiding this comment

k8s-github-robot commented Dec 16, 2017

k8s-ci-robot commented Dec 16, 2017 • edited Loading

k8s-github-robot commented Dec 16, 2017

ingvagabund commented Nov 22, 2017 •

edited

Loading

ingvagabund Dec 15, 2017 •

edited

Loading

k8s-ci-robot commented Dec 16, 2017 •

edited

Loading