Fix subpath disruptive tests #113853

jsafrane · 2022-11-11T13:05:28Z

What type of PR is this?

/kind bug
/kind failing-test

What this PR does / why we need it:

#113596 introduced a new check with hardcoded /mnt/volume for a mounted vulume, but subpath tests use /test-volume in their pods. Add volumePath parameter to all disruptive checks, so subpath and disruptive tests can use their own paths to test volumes in their respective Pods.

Which issue(s) this PR fixes:

Fixes #113849

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Add volumePath parameter to all disruptive checks, so subpath tests can use "/test-volume" and disruptive tests can use "/mnt/volume1" for their respective Pods.

jsafrane · 2022-11-11T13:10:31Z

/test disruptive
Looking for a disruptive job for storage

k8s-ci-robot · 2022-11-11T13:10:34Z

@jsafrane: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test pull-kubernetes-conformance-kind-ga-only-parallel
/test pull-kubernetes-coverage-unit
/test pull-kubernetes-dependencies
/test pull-kubernetes-dependencies-go-canary
/test pull-kubernetes-e2e-gce
/test pull-kubernetes-e2e-gce-100-performance
/test pull-kubernetes-e2e-gce-big-performance
/test pull-kubernetes-e2e-gce-canary
/test pull-kubernetes-e2e-gce-network-proxy-http-connect
/test pull-kubernetes-e2e-gce-no-stage
/test pull-kubernetes-e2e-gce-scale-performance-manual
/test pull-kubernetes-e2e-gce-ubuntu-containerd
/test pull-kubernetes-e2e-gce-ubuntu-containerd-canary
/test pull-kubernetes-e2e-kind
/test pull-kubernetes-e2e-kind-ipv6
/test pull-kubernetes-integration
/test pull-kubernetes-integration-go-canary
/test pull-kubernetes-kubemark-e2e-gce-scale
/test pull-kubernetes-node-e2e-containerd
/test pull-kubernetes-typecheck
/test pull-kubernetes-unit
/test pull-kubernetes-unit-go-canary
/test pull-kubernetes-update
/test pull-kubernetes-verify
/test pull-kubernetes-verify-go-canary
/test pull-kubernetes-verify-govet-levee

The following commands are available to trigger optional jobs:

/test check-dependency-stats
/test pull-ci-kubernetes-unit-windows
/test pull-e2e-gce-cloud-provider-disabled
/test pull-kubernetes-conformance-image-test
/test pull-kubernetes-conformance-kind-ga-only
/test pull-kubernetes-conformance-kind-ipv6-parallel
/test pull-kubernetes-cos-cgroupv1-containerd-node-e2e
/test pull-kubernetes-cos-cgroupv1-containerd-node-e2e-features
/test pull-kubernetes-cos-cgroupv2-containerd-node-e2e
/test pull-kubernetes-cos-cgroupv2-containerd-node-e2e-features
/test pull-kubernetes-cos-cgroupv2-containerd-node-e2e-serial
/test pull-kubernetes-cross
/test pull-kubernetes-e2e-capz-azure-disk
/test pull-kubernetes-e2e-capz-azure-disk-vmss
/test pull-kubernetes-e2e-capz-azure-file
/test pull-kubernetes-e2e-capz-azure-file-vmss
/test pull-kubernetes-e2e-capz-conformance
/test pull-kubernetes-e2e-capz-ha-control-plane
/test pull-kubernetes-e2e-capz-windows-containerd
/test pull-kubernetes-e2e-containerd-gce
/test pull-kubernetes-e2e-gce-alpha-features
/test pull-kubernetes-e2e-gce-correctness
/test pull-kubernetes-e2e-gce-csi-serial
/test pull-kubernetes-e2e-gce-device-plugin-gpu
/test pull-kubernetes-e2e-gce-kubetest2
/test pull-kubernetes-e2e-gce-network-proxy-grpc
/test pull-kubernetes-e2e-gce-storage-disruptive
/test pull-kubernetes-e2e-gce-storage-slow
/test pull-kubernetes-e2e-gce-storage-snapshot
/test pull-kubernetes-e2e-gce-ubuntu-containerd-serial
/test pull-kubernetes-e2e-gci-gce-autoscaling
/test pull-kubernetes-e2e-gci-gce-ingress
/test pull-kubernetes-e2e-gci-gce-ipvs
/test pull-kubernetes-e2e-inplace-pod-resize-containerd-main-v2
/test pull-kubernetes-e2e-kind-canary
/test pull-kubernetes-e2e-kind-dual-canary
/test pull-kubernetes-e2e-kind-ipv6-canary
/test pull-kubernetes-e2e-kind-ipvs-dual-canary
/test pull-kubernetes-e2e-kind-multizone
/test pull-kubernetes-e2e-kops-aws
/test pull-kubernetes-e2e-kubelet-credential-provider
/test pull-kubernetes-e2e-ubuntu-gce-network-policies
/test pull-kubernetes-kind-dra
/test pull-kubernetes-kubemark-e2e-gce-big
/test pull-kubernetes-local-e2e
/test pull-kubernetes-node-crio-cgrpv2-e2e
/test pull-kubernetes-node-crio-cgrpv2-e2e-kubetest2
/test pull-kubernetes-node-crio-e2e
/test pull-kubernetes-node-crio-e2e-kubetest2
/test pull-kubernetes-node-e2e-containerd-alpha-features
/test pull-kubernetes-node-e2e-containerd-features
/test pull-kubernetes-node-e2e-containerd-features-kubetest2
/test pull-kubernetes-node-e2e-containerd-kubetest2
/test pull-kubernetes-node-kubelet-credential-provider
/test pull-kubernetes-node-kubelet-serial-containerd
/test pull-kubernetes-node-kubelet-serial-containerd-kubetest2
/test pull-kubernetes-node-kubelet-serial-cpu-manager
/test pull-kubernetes-node-kubelet-serial-cpu-manager-kubetest2
/test pull-kubernetes-node-kubelet-serial-crio-cgroupv1
/test pull-kubernetes-node-kubelet-serial-crio-cgroupv2
/test pull-kubernetes-node-kubelet-serial-hugepages
/test pull-kubernetes-node-kubelet-serial-memory-manager
/test pull-kubernetes-node-kubelet-serial-topology-manager
/test pull-kubernetes-node-kubelet-serial-topology-manager-kubetest2
/test pull-kubernetes-node-memoryqos-cgrpv2
/test pull-kubernetes-node-swap-fedora
/test pull-kubernetes-node-swap-fedora-serial
/test pull-kubernetes-node-swap-ubuntu-serial
/test pull-kubernetes-unit-experimental
/test pull-publishing-bot-validate

Use /test all to run the following jobs that were automatically triggered:

pull-kubernetes-conformance-kind-ga-only-parallel
pull-kubernetes-conformance-kind-ipv6-parallel
pull-kubernetes-dependencies
pull-kubernetes-e2e-gce-100-performance
pull-kubernetes-e2e-gce-csi-serial
pull-kubernetes-e2e-gce-storage-slow
pull-kubernetes-e2e-gce-storage-snapshot
pull-kubernetes-e2e-gce-ubuntu-containerd
pull-kubernetes-e2e-kind
pull-kubernetes-e2e-kind-ipv6
pull-kubernetes-integration
pull-kubernetes-node-e2e-containerd
pull-kubernetes-typecheck
pull-kubernetes-unit
pull-kubernetes-verify
pull-kubernetes-verify-govet-levee

In response to this:

/test disruptive
Looking for a disruptive job for storage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jsafrane · 2022-11-11T13:10:54Z

/test pull-kubernetes-e2e-gce-storage-disruptive

jsafrane · 2022-11-11T13:13:40Z

/priority important-soon

aojea · 2022-11-14T11:04:58Z

/test pull-kubernetes-e2e-gce-storage-disruptive

is like it failed much more tests that previous execution, how can we be sure this is ok?

/test pull-kubernetes-e2e-gce-storage-slow

/assign @msau42 @pohly

gnufied · 2022-11-14T19:20:11Z

This testsuite appears to be still failing:

{Nov 14 11:32:37.226: PVC, PV creation failed: PVC Create API error: PersistentVolumeClaim "pvc-tmcpl" is invalid: spec.accessModes: Unsupported value: "ReadWriteOncePod": supported values: "ReadOnlyMany", "ReadWriteMany", "ReadWriteOnce" failed test/e2e/storage/framework/volume_resource.go:271
k8s.io/kubernetes/test/e2e/storage/framework.createPVCPV(0xc001818f00, {0x75ae935, 0x5}, 0xc0007f5760, 0xc000844010, {0x75be2ff, 0xa}, {0xc001a35ba0, 0x1, 0x1})
	test/e2e/storage/framework/volume_resource.go:271 +0x369
k8s.io/kubernetes/test/e2e/storage/framework.CreateVolumeResourceWithAccessModes({0x7fbcca8, 0xc001724900}, 0xc001a7d4a0, {{0x7682e01, 0x27}, {0x0, 0x0}, {0x75da374, 0x10}, {0x0, ...}, ...}, ...)
	test/e2e/storage/framework/volume_resource.go:83 +0x10e5
k8s.io/kubernetes/test/e2e/storage/testsuites.(*disruptiveTestSuite).DefineTests.func1({0xc001a35ba0?, 0x1, 0x1})
	test/e2e/storage/testsuites/disruptive.go:108 +0x25a
k8s.io/kubernetes/test/e2e/storage/testsuites.(*disruptiveTestSuite).DefineTests.func8.1()
	test/e2e/storage/testsuites/disruptive.go:236 +0xc3}
[open stderropen_in_new](https://prow.k8s.io/spyglass/lens/junit/iframe?req=%7B%22artifacts%22%3A%5B%22artifacts%2Fjunit_01.xml%22%2C%22artifacts%2Fjunit_runner.xml%22%5D%2C%22index%22%3A2%2C%22src%22%3A%22gs%2Fkubernetes-jenkins%2Fpr-logs%2Fpull%2F113853%2Fpull-kubernetes-e2e-gce-storage-disruptive%2F1592111346969743360%22%7D&topURL=https%3A//prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/113853/pull-kubernetes-e2e-gce-storage-disruptive/1592111346969743360/&lensIndex=2#)

gnufied · 2022-11-14T19:21:31Z

I wonder if test names are generated dynamically, they aren't skipped in non-alpha clusters and hence these tests are perma failing. @jsafrane should we revert the e2e part of your PR, while we sort this out?

dims · 2022-11-15T02:35:37Z

/approve
/lgtm
/retest
/milestone v1.26
/priority critical-urgent
/kind failing-test
/triage accepted

k8s-ci-robot · 2022-11-15T02:36:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~test/e2e/framework/OWNERS~~ [dims]
~~test/e2e/storage/OWNERS~~ [dims,jsafrane]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

dims · 2022-11-15T02:36:11Z

/hold

@jsafrane please remove hold when ready

jsafrane · 2022-11-15T10:51:47Z

I wonder if test names are generated dynamically, they aren't skipped in non-alpha clusters and hence these tests are perma failing.

But... pull-kubernetes-e2e-gce-storage-disruptive does not install an alpha cluster. The CI job misses --skip=Feature:.

@jsafrane should we revert the e2e part of your PR, while we sort this out?

Last attempt: kubernetes/test-infra#28008

jsafrane · 2022-11-15T12:28:17Z

/retest

gnufied · 2022-11-15T14:45:34Z

/lgtm

gnufied · 2022-11-15T14:46:12Z

( @dims hope you don't mind cancelling the hold) - All tests are passing now.

/hold cancel

dims · 2022-11-15T14:49:01Z

( @dims hope you don't mind cancelling the hold) - All tests are passing now.

🚢 🚢 🚢 🚢 🚢 🚢

jsafrane · 2022-11-15T15:27:04Z

/hold cancel
e2e-gce-storage-disruptive is green

Fix subpath disruptive tests

3fdd4ca

Add volumePath parameter to all disruptive checks, so subpath tests can use "/test-volume" and disruptive tests can use "/mnt/volume1" for their respective Pods.

k8s-ci-robot requested review from andrewsykim and pohly November 11, 2022 13:06

k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 11, 2022

k8s-ci-robot assigned msau42 and pohly Nov 14, 2022

k8s-ci-robot added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label Nov 15, 2022

k8s-ci-robot added this to the v1.26 milestone Nov 15, 2022

k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 15, 2022

k8s-ci-robot assigned dims Nov 15, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 15, 2022

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 15, 2022

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 15, 2022

jsafrane mentioned this pull request Nov 15, 2022

Skip Feature: tests in e2e-gce-storage-disruptive kubernetes/test-infra#28008

Merged

k8s-ci-robot assigned gnufied Nov 15, 2022

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 15, 2022

k8s-ci-robot merged commit d34bdee into kubernetes:master Nov 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix subpath disruptive tests #113853

Fix subpath disruptive tests #113853

jsafrane commented Nov 11, 2022 •

edited

jsafrane commented Nov 11, 2022

k8s-ci-robot commented Nov 11, 2022

jsafrane commented Nov 11, 2022

jsafrane commented Nov 11, 2022

aojea commented Nov 14, 2022

gnufied commented Nov 14, 2022

gnufied commented Nov 14, 2022

dims commented Nov 15, 2022

k8s-ci-robot commented Nov 15, 2022

dims commented Nov 15, 2022

jsafrane commented Nov 15, 2022

jsafrane commented Nov 15, 2022

gnufied commented Nov 15, 2022

gnufied commented Nov 15, 2022 •

edited

dims commented Nov 15, 2022

jsafrane commented Nov 15, 2022

Fix subpath disruptive tests #113853

Fix subpath disruptive tests #113853

Conversation

jsafrane commented Nov 11, 2022 • edited

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

jsafrane commented Nov 11, 2022

k8s-ci-robot commented Nov 11, 2022

jsafrane commented Nov 11, 2022

jsafrane commented Nov 11, 2022

aojea commented Nov 14, 2022

gnufied commented Nov 14, 2022

gnufied commented Nov 14, 2022

dims commented Nov 15, 2022

k8s-ci-robot commented Nov 15, 2022

dims commented Nov 15, 2022

jsafrane commented Nov 15, 2022

jsafrane commented Nov 15, 2022

gnufied commented Nov 15, 2022

gnufied commented Nov 15, 2022 • edited

dims commented Nov 15, 2022

jsafrane commented Nov 15, 2022

jsafrane commented Nov 11, 2022 •

edited

gnufied commented Nov 15, 2022 •

edited