Add flag to prune PASSED subtests in junit xml and have top level tests #117029

Rajalakshmi-Girish · 2023-03-31T05:28:46Z

The CI which runs k8s unit tests has a huge no.of rows while it uses https://github.com/kubernetes/kubernetes/blob/master/hack/make-rules/test.sh and generates junit xml file in artifacts.

This large no. of rows hinders the display of complete data in test grids

Ref: kubernetes/test-infra#24779
kubernetes/test-infra#23425

As the purpose of the test grid in certain cases would be to analyze the failing/flaking tests, it shall be useful to have an option in prune-junit-xml to prune PASSED test cases and display only failed test cases.

k8s-ci-robot · 2023-03-31T05:28:50Z

Please note that we're already in Test Freeze for the release-1.27 branch. This means every merged PR will be automatically fast-forwarded via the periodic ci-fast-forward job to the release branch of the upcoming v1.27.0 release.

Fast forwards are scheduled to happen every 6 hours, whereas the most recent run was: Fri Mar 31 04:32:14 UTC 2023.

k8s-ci-robot · 2023-03-31T05:28:56Z

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Rajalakshmi-Girish · 2023-03-31T05:37:03Z

/kind bug
/sig testing

liggitt · 2023-03-31T05:39:41Z

is the expectation that all/most testgrids would use this option by default? losing info about green runs will make it hard to tell the difference between a test running and being skipped, as well as losing run-time information testgrid surfaces as graphs of times

BenTheElder · 2023-03-31T06:03:41Z

Can we collapse to e.g. package level results instead?

liggitt · 2023-03-31T06:04:57Z

Can we collapse to e.g. package level results instead?

or even package+top-level test ... a lot of the unit/integration test bulk comes from tests with ~hundreds of programmatically generated subtests

Rajalakshmi-Girish · 2023-03-31T09:12:42Z

Can we collapse to e.g. package level results instead?

or even package+top-level test ... a lot of the unit/integration test bulk comes from tests with ~hundreds of programmatically generated subtests

I have changed the logic to display the package + top-level tests. The subtests would be shown only in cases when there is a failure.

This has drastically brought down the no.of rows from ~39k to ~9k. Also, many stale rows that are generated by Tests like apiserver/pkg/cel.TestEscaping would be got rid of in testgrid.

Without Pruning https://prow.ppc64le-cloud.org/view/gs/ppc64le-kubernetes/logs/periodic-kubernetes-unit-test-ppc64le/1641471234464026624
With -prune-passed-subtests=true https://prow.ppc64le-cloud.org/view/gs/ppc64le-kubernetes/logs/periodic-kubernetes-unit-test-ppc64le/1641199440779808768

@BenTheElder @liggitt Can you PTAL now?

dims · 2023-03-31T10:17:01Z

/release-note-none

cc @aojea @pohly

cmd/prune-junit-xml/prunexml.go

pohly

The change itself looks good to me.

However, should this PR also enable the new feature? Right now it doesn't, because default is false and the command line flag is not getting added to the test scripts which invoke prune-junit.

I think this PR should enable pruning. The simplest way to do that would be to change the flag default. Only Kubernetes uses it, so we don't need to worry about breaking someone other than ourselves.

BenTheElder

/lgtm
+1 to flip the default for prune-tests to true though

k8s-ci-robot · 2023-05-25T17:27:24Z

LGTM label has been added.

Git tree hash: 0277c75cade10fc65fbf1fcd64cbe94ea5b6b9ff

BenTheElder · 2023-05-25T17:27:37Z

/approve

Rajalakshmi-Girish · 2023-06-06T07:04:20Z

Messed up while rebasing and resolving conflicts :(
Please see #118487

Rajalakshmi-Girish · 2023-06-07T10:29:40Z

@pohly ^^

pohly

/lgtm

pohly · 2023-06-07T11:02:31Z

cmd/prune-junit-xml/prunexml.go


 	"k8s.io/kubernetes/third_party/forked/gotestsum/junitxml"
 )

 func main() {
 	maxTextSize := flag.Int("max-text-size", 1, "maximum size of attribute or text (in MB)")
+	pruneTests := flag.Bool("prune-tests", true,


It's on by default now, as suggested in #117029 (review)

k8s-ci-robot · 2023-06-07T11:03:11Z

LGTM label has been added.

Git tree hash: 7f3495dc1567f3a2b24b133d1a3dec8ba2dc106c

pohly · 2023-06-07T11:36:46Z

Looks like we need one more approval.

/assign @dims

dims · 2023-06-07T12:20:08Z

/approve

thanks @Rajalakshmi-Girish

k8s-ci-robot · 2023-06-07T12:20:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, dims, pohly, Rajalakshmi-Girish

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cmd/prune-junit-xml/OWNERS~~ [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 31, 2023

k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Mar 31, 2023

k8s-ci-robot requested review from liggitt and thockin March 31, 2023 05:29

liggitt added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Mar 31, 2023

Rajalakshmi-Girish force-pushed the prune-junit-xml branch 3 times, most recently from d1d8343 to 67635bd Compare March 31, 2023 08:51

Rajalakshmi-Girish changed the title ~~Add flag to prune PASSED tests in junit xml and have only failed testcases~~ Add flag to prune PASSED subtests in junit xml and have top level tests Mar 31, 2023

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 31, 2023

pohly reviewed Apr 3, 2023

View reviewed changes

cmd/prune-junit-xml/prunexml.go Outdated Show resolved Hide resolved

pohly reviewed May 25, 2023

View reviewed changes

BenTheElder reviewed May 25, 2023

View reviewed changes

k8s-ci-robot assigned BenTheElder May 25, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 25, 2023

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 26, 2023

k8s-ci-robot removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jun 6, 2023

k8s-ci-robot requested a review from BenTheElder June 6, 2023 06:01

k8s-ci-robot added the do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. label Jun 6, 2023

Rajalakshmi-Girish mentioned this pull request Jun 6, 2023

prune's xml files to display only top level tests and failed sub-tests #118487

Closed

Rajalakshmi-Girish closed this Jun 6, 2023

prune's xml files to display only top level tests and failed sub-tests

dddbecb

Rajalakshmi-Girish reopened this Jun 7, 2023

Rajalakshmi-Girish force-pushed the prune-junit-xml branch from 29cf8e5 to dddbecb Compare June 7, 2023 07:16

k8s-ci-robot removed the do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. label Jun 7, 2023

pohly approved these changes Jun 7, 2023

View reviewed changes

k8s-ci-robot assigned pohly Jun 7, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 7, 2023

k8s-ci-robot assigned dims Jun 7, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 7, 2023

k8s-ci-robot merged commit 7ef5ec4 into kubernetes:master Jun 7, 2023
12 checks passed

k8s-ci-robot added this to the v1.28 milestone Jun 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flag to prune PASSED subtests in junit xml and have top level tests #117029

Add flag to prune PASSED subtests in junit xml and have top level tests #117029

Rajalakshmi-Girish commented Mar 31, 2023

k8s-ci-robot commented Mar 31, 2023

k8s-ci-robot commented Mar 31, 2023

Rajalakshmi-Girish commented Mar 31, 2023

liggitt commented Mar 31, 2023

BenTheElder commented Mar 31, 2023

liggitt commented Mar 31, 2023 •

edited

Rajalakshmi-Girish commented Mar 31, 2023

dims commented Mar 31, 2023

pohly left a comment

BenTheElder left a comment

k8s-ci-robot commented May 25, 2023

BenTheElder commented May 25, 2023

Rajalakshmi-Girish commented Jun 6, 2023

Rajalakshmi-Girish commented Jun 7, 2023

pohly left a comment

pohly Jun 7, 2023

k8s-ci-robot commented Jun 7, 2023

pohly commented Jun 7, 2023

dims commented Jun 7, 2023

k8s-ci-robot commented Jun 7, 2023

Add flag to prune PASSED subtests in junit xml and have top level tests #117029

Add flag to prune PASSED subtests in junit xml and have top level tests #117029

Conversation

Rajalakshmi-Girish commented Mar 31, 2023

k8s-ci-robot commented Mar 31, 2023

k8s-ci-robot commented Mar 31, 2023

Rajalakshmi-Girish commented Mar 31, 2023

liggitt commented Mar 31, 2023

BenTheElder commented Mar 31, 2023

liggitt commented Mar 31, 2023 • edited

Rajalakshmi-Girish commented Mar 31, 2023

dims commented Mar 31, 2023

pohly left a comment

Choose a reason for hiding this comment

BenTheElder left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented May 25, 2023

BenTheElder commented May 25, 2023

Rajalakshmi-Girish commented Jun 6, 2023

Rajalakshmi-Girish commented Jun 7, 2023

pohly left a comment

Choose a reason for hiding this comment

pohly Jun 7, 2023

Choose a reason for hiding this comment

k8s-ci-robot commented Jun 7, 2023

pohly commented Jun 7, 2023

dims commented Jun 7, 2023

k8s-ci-robot commented Jun 7, 2023

liggitt commented Mar 31, 2023 •

edited