ci(test): skip skipped tests from the Test JUnit output #8084

v1v · 2025-05-06T08:34:59Z

What does this PR do?

Bump the gotestsum version and eemove skipped tests from the aggregated report.

Why is it important?

The integration tests reporting have a few non-critical reporting issues:

Each test step has its own XML and HTML report in atrifacts (Build example)
The reports contain many of skipped tests (example)

That affects:

General test report readability
Buildkite Test Suites show wrong AVG duration and failure percentage

Checklist

I have read and understood the pull request guidelines of this project.
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Run make and then .

I compared this test-result from main vs this PR

vs

Related issues

gotestyourself/gotestsum#478 has been merged and https://github.com/gotestyourself/gotestsum/releases/tag/v1.12.2 has been released

Questions to ask yourself

How are we going to support this in production?
How are we going to measure its adoption?
How are we going to debug this?
What are the metrics I should take care of?
...

mergify · 2025-05-06T08:35:34Z

This pull request does not have a backport label. Could you fix it @v1v? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
backport-active-all is the label that automatically backports to all active branches.
backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

pkoutsovasilis

Should we have here the --junitfile-hide-skipped-tests argument?

v1v · 2025-05-06T16:14:00Z

Should we have here the --junitfile-hide-skipped-tests argument?

good catch, I somehow thought it was using the mage specific implementation. Thanks!

See 77dcc3f (#8084)

mergify · 2025-05-07T20:14:58Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feature/gotestsum-skip upstream/feature/gotestsum-skip
git merge upstream/main
git push upstream feature/gotestsum-skip

* upstream/main: Guard against `nil` pointer dereference (elastic#8107) Generate NOTICE.txt with only modules used by binaries (elastic#8053) Retry enrollment requests when an error is returned, add enrollment timeout (elastic#8056) Changelog for 8.17.6 version (elastic#8062) (elastic#8106) [main][Automation] Update versions (elastic#8098) Allow using beats receivers for self-monitoring (elastic#8031) Adding new configuration setting: `agent.upgrade.rollback.window` (elastic#8065) [Integration Testing] Allow tests to declare themselves as needing a FIPS environment (elastic#8083) fix(agentless): overcome SIGPIPE in agentless promotion pipeline (elastic#8094) ksm autosharing integration configuration update (elastic#8086)

pkoutsovasilis · 2025-05-08T10:08:20Z

@v1v the code changes LGTM but maybe also @pazone should have a look and somehow cross-check that now that the skipped tests are not reported buildkite statistics do not have the reported inconsistencies? Also don't we want to backport this to all active branches?

mergify · 2025-05-08T20:50:32Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feature/gotestsum-skip upstream/feature/gotestsum-skip
git merge upstream/main
git push upstream feature/gotestsum-skip

elasticmachine · 2025-05-09T07:32:34Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

pazone

LGTM. It's now much easier to read the test reports
Should we replace Replace() with ReplaceAll() in gotest.go within this PR?

pkoutsovasilis

@v1v we need to add the new --junitfile-hide-skipped-tests flag also here right?

v1v · 2025-05-16T00:05:41Z

@v1v we need to add the new --junitfile-hide-skipped-tests flag also here right?

Thanks! see c41303b (#8084)

elastic-sonarqube · 2025-05-16T01:18:51Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

elasticmachine · 2025-05-16T23:44:04Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: e75dba2

Failed CI Steps

History

💚 Build #21106 succeeded 030f2ea
💚 Build #21065 succeeded 4339920
💛 Build #20969 was flaky 77dcc3f
💔 Build #20941 failed 5cecb16
💔 Build #20930 failed 0df8716

cc @v1v

github-actions · 2025-05-19T08:06:34Z

@Mergifyio backport 9.0

mergify · 2025-05-19T08:06:47Z

backport 9.0

✅ Backports have been created

#8167 [9.0] (backport #8084) ci(test): skip skipped tests from the Test JUnit output has been created for branch 9.0 but encountered conflicts

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1

v1v · 2025-05-19T08:08:56Z

@Mergifyio backport 8.18

mergify · 2025-05-19T08:09:05Z

backport 8.18

✅ Backports have been created

#8169 [8.18] (backport #8084) ci(test): skip skipped tests from the Test JUnit output has been created for branch 8.18 but encountered conflicts

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum

…nit output (#8168)

…nit output (#8169)

…it output (#8167)

pkoutsovasilis · 2025-06-12T05:51:08Z

@Mergifyio backport 8.17

mergify · 2025-06-12T05:51:17Z

backport 8.17

✅ Backports have been created

#8461 [8.17] (backport #8084) ci(test): skip skipped tests from the Test JUnit output has been created for branch 8.17 but encountered conflicts

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/buildkite-integration-tests.sh # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum

…nit output (#8461) * ci(test): skip skipped tests from the Test JUnit output (#8084) (cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/buildkite-integration-tests.sh # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum * fix: resolve conflicts * fix: update NOTICE.txt --------- Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>

ci(test): skip skipped tests from the Test JUnit output

0df8716

v1v self-assigned this May 6, 2025

v1v added skip-changelog backport-active-9 Automated backport with mergify to all the active 9.[0-9]+ branches backport-8.19 Automated backport to the 8.19 branch labels May 6, 2025

fix

5cecb16

v1v marked this pull request as ready for review May 6, 2025 12:52

v1v requested review from a team as code owners May 6, 2025 12:52

v1v requested review from blakerouse and pkoutsovasilis May 6, 2025 12:52

pkoutsovasilis reviewed May 6, 2025

View reviewed changes

add missed --junitfile-hide-skipped-tests in the gotestsum calls

77dcc3f

Merge branch 'main' into feature/gotestsum-skip

030f2ea

pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label May 9, 2025

pazone previously approved these changes May 12, 2025

View reviewed changes

v1v requested a review from pkoutsovasilis May 15, 2025 14:16

pkoutsovasilis reviewed May 15, 2025

View reviewed changes

add missing flag

c41303b

v1v dismissed pazone’s stale review via c41303b May 16, 2025 00:05

Merge branch 'main' into feature/gotestsum-skip

e75dba2

pkoutsovasilis approved these changes May 16, 2025

View reviewed changes

v1v merged commit 05289da into elastic:main May 19, 2025
9 of 12 checks passed

mergify bot pushed a commit that referenced this pull request May 19, 2025

ci(test): skip skipped tests from the Test JUnit output (#8084)

e716b20

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum

mergify bot mentioned this pull request May 19, 2025

[9.0] (backport #8084) ci(test): skip skipped tests from the Test JUnit output #8167

Merged

8 tasks

mergify bot pushed a commit that referenced this pull request May 19, 2025

ci(test): skip skipped tests from the Test JUnit output (#8084)

f02e2cb

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1

mergify bot mentioned this pull request May 19, 2025

[8.19] (backport #8084) ci(test): skip skipped tests from the Test JUnit output #8168

Merged

8 tasks

mergify bot pushed a commit that referenced this pull request May 19, 2025

ci(test): skip skipped tests from the Test JUnit output (#8084)

5ff1204

(cherry picked from commit 05289da) # Conflicts: # .buildkite/scripts/integration-tests.ps1 # go.mod # go.sum

mergify bot mentioned this pull request May 19, 2025

[8.18] (backport #8084) ci(test): skip skipped tests from the Test JUnit output #8169

Merged

8 tasks

v1v pushed a commit that referenced this pull request May 19, 2025

[8.19] (backport #8084) ci(test): skip skipped tests from the Test JU…

cc24929

…nit output (#8168)

v1v pushed a commit that referenced this pull request May 19, 2025

[8.18] (backport #8084) ci(test): skip skipped tests from the Test JU…

ba34e2d

…nit output (#8169)

v1v pushed a commit that referenced this pull request May 20, 2025

[9.0] (backport #8084) ci(test): skip skipped tests from the Test JUn…

b83a7ee

…it output (#8167)

pkoutsovasilis mentioned this pull request Jun 11, 2025

[8.17] (backport #8396) [tests] split up serverless and resource leaks integration tests #8455

Merged

8 tasks

mergify bot mentioned this pull request Jun 12, 2025

[8.17] (backport #8084) ci(test): skip skipped tests from the Test JUnit output #8461

Merged

8 tasks

ci(test): skip skipped tests from the Test JUnit output #8084

ci(test): skip skipped tests from the Test JUnit output #8084

Uh oh!

Conversation

v1v commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

Questions to ask yourself

Uh oh!

mergify bot commented May 6, 2025

Uh oh!

pkoutsovasilis left a comment

Choose a reason for hiding this comment

Uh oh!

v1v commented May 6, 2025

Uh oh!

mergify bot commented May 7, 2025

Uh oh!

pkoutsovasilis commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented May 8, 2025

Uh oh!

elasticmachine commented May 9, 2025

Uh oh!

pazone left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pkoutsovasilis left a comment

Choose a reason for hiding this comment

Uh oh!

v1v commented May 16, 2025

Uh oh!

elastic-sonarqube bot commented May 16, 2025

Quality Gate passed

Uh oh!

elasticmachine commented May 16, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

History

Uh oh!

Uh oh!

github-actions bot commented May 19, 2025

Uh oh!

mergify bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

v1v commented May 19, 2025

Uh oh!

mergify bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

pkoutsovasilis commented Jun 12, 2025

Uh oh!

mergify bot commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Uh oh!

v1v commented May 6, 2025 •

edited

Loading

pkoutsovasilis commented May 8, 2025 •

edited

Loading

pazone left a comment •

edited

Loading

mergify bot commented May 19, 2025 •

edited

Loading

mergify bot commented May 19, 2025 •

edited

Loading

mergify bot commented Jun 12, 2025 •

edited

Loading