Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

e2e: test labels #121394

Merged
merged 3 commits into from Nov 3, 2023
Merged

e2e: test labels #121394

merged 3 commits into from Nov 3, 2023

Conversation

pohly
Copy link
Contributor

@pohly pohly commented Oct 20, 2023 •
edited

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Adding tags with the new framework helper functions also adds them as Ginkgo labels. Labels enable much more flexible filtering of specs. The goal is to register all tests this way, with no or only minimal changes to the full test names, so existing regular expressions should continue to work.

There's one exception: a space now gets inserted around tags, which wasn't always the case before. If a regular expression in a Prow job is sensitive to white space, then it will not match anymore.

Which issue(s) this PR fixes:

Related-to: kubernetes/enhancements#3041

The KEP discusses semantic changes. Those are not part of this PR. Instead, the goal is to remain as close as possible to the current test names.

Special notes for your reviewer:

A subset was merged in #121391. Announced in https://groups.google.com/a/kubernetes.io/g/dev/c/DTFEng143NY

Does this PR introduce a user-facing change?

NONE

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Oct 20, 2023
This was referenced Oct 20, 2023
Merged
Open
@k8s-ci-robot k8s-ci-robot added area/conformance Issues or PRs related to kubernetes conformance tests area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/network-policy Issues or PRs related to Network Policy subproject area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/autoscaling Categorizes an issue or PR as relevant to SIG Autoscaling. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 20, 2023
@pohly
Copy link
Contributor Author

pohly commented Nov 1, 2023

@aojea : rebased, please lgtm again.

@pohly
Copy link
Contributor Author

pohly commented Nov 1, 2023

/remove-sig api-machinery

@k8s-ci-robot k8s-ci-robot removed the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Nov 1, 2023
@aojea
Copy link
Member

aojea commented Nov 1, 2023

it seems I broke you ERROR: test/e2e/network/service_cidrs.go:38: [Feature:ServiceCIDRs] in plain text is deprecated and must be added through WithFeature(ServiceCIDRs) instead 😄

sorry about that

@pohly
e2e: use framework labels
f2cfbf4 
This changes the text registration so that tags for which the framework has a
dedicated API (features, feature gates, slow, serial, etc.) those APIs are
used.

Arbitrary, custom tags are still left in place for now.
@pohly
e2e framework: validate test definitions
338fe7a 
This checks that the With* label functions are used instead of the previous
inline tags. To catch strings passed to Ginkgo directly instead of the
framework wrapper functions, the final test specs are checked.
@pohly
Copy link
Contributor Author

pohly commented Nov 1, 2023

Hey, it's working as intended - good check! 😁

Let me fix that... would have been trivial if I had remembered to run hack/verify-e2e-suites.sh:

ERROR: E2E test suite invocation failed for test/e2e.
   ERROR: E2E suite initialization was faulty, these errors must be fixed:
   ERROR: test/e2e/network/service_cidrs.go:38: [Feature:ServiceCIDRs] in plain text is deprecated and must be added through WithFeature(ServiceCIDRs) instead
E2E suite test/e2e_kubeadm passed.
ERROR: E2E test suite invocation failed for test/e2e_node.
   ERROR: E2E suite initialization was faulty, these errors must be fixed:
   ERROR: test/e2e_node/container_lifecycle_test.go:2964: [Serial] in plain text is deprecated and must be added through WithSerial instead

Fixed: https://github.com/kubernetes/kubernetes/compare/2710abf664524c3f544bf5c2410669075febd72b..338fe7ad55cad858284d4c96df86d3cee9c7d29c

@k8s-ci-robot k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Nov 1, 2023
@pohly
Copy link
Contributor Author

pohly commented Nov 1, 2023

/test pull-kubernetes-conformance-kind-ga-only-parallel

@SergeyKanzhelev
Copy link
Member

/assign

@SergeyKanzhelev SergeyKanzhelev moved this from Triage to Issues - In progress in SIG Node CI/Test Board Nov 1, 2023
@SergeyKanzhelev SergeyKanzhelev moved this from Issues - In progress to PRs - Needs Reviewer in SIG Node CI/Test Board Nov 1, 2023
@pohly
Copy link
Contributor Author

pohly commented Nov 1, 2023

/retest

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 1, 2023
@k8s-ci-robot
Copy link
Contributor

@pohly: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-e2e-ubuntu-gce-network-policies 338fe7a link false /test pull-kubernetes-e2e-ubuntu-gce-network-policies
pull-kubernetes-conformance-image-test 338fe7a link false /test pull-kubernetes-conformance-image-test

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@seans3
Copy link
Contributor

seans3 commented Nov 1, 2023

/triage accepted
/priority important-soon

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 1, 2023
@aojea
Copy link
Member

aojea commented Nov 3, 2023

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 3, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: ecbc08a4c38706e5bb76db1a22cd06d39ffb7800

@k8s-ci-robot k8s-ci-robot merged commit 66e20ff into kubernetes:master Nov 3, 2023
28 of 30 checks passed
SIG Node CI/Test Board automation moved this from PRs - Needs Reviewer to Done Nov 3, 2023
SIG Node PR Triage automation moved this from WIP to Done Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aojea aojea aojea left review comments

@abhiraut abhiraut Awaiting requested review from abhiraut

@andrewsykim andrewsykim Awaiting requested review from andrewsykim

@dims dims Awaiting requested review from dims

Assignees

@dims dims

@aojea aojea

@SergeyKanzhelev SergeyKanzhelev

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance Issues or PRs related to kubernetes conformance tests area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/network-policy Issues or PRs related to Network Policy subproject area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/autoscaling Categorizes an issue or PR as relevant to SIG Autoscaling. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage. sig/testing Categorizes an issue or PR as relevant to SIG Testing. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Apps
Archived in project
SIG Auth
Archived in project
SIG CLI
Archived in project
SIG Node CI/Test Board
Done
SIG Node PR Triage
Done
[sig-release] Bug Triage
Archived in project
[sig-windows] PR tracking
Status: Done
Milestone
v1.29
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@pohly @k8s-ci-robot @leilajal @dims @aojea @MaryamTavakkoli @SergeyKanzhelev @seans3