Skip to content

ci: enable Codecov Test Analytics across all test jobs#5207

Merged
Yicong-Huang merged 1 commit into
apache:mainfrom
Yicong-Huang:ci/codecov-test-analytics
May 25, 2026
Merged

ci: enable Codecov Test Analytics across all test jobs#5207
Yicong-Huang merged 1 commit into
apache:mainfrom
Yicong-Huang:ci/codecov-test-analytics

Conversation

@Yicong-Huang
Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Enable Codecov Test Analytics for every stack we already upload coverage from. Each stack now emits JUnit-XML alongside its existing coverage report, and a second codecov/codecov-action step per job uploads it with report_type: test_results.

  • ScalaTest: ThisBuild / Test / testOptions += -u target/test-reports in the root build.sbt.
  • Vitest unit + browser configs: reporters: ["default", ["junit", { outputFile: ... }]].
  • pytest (python job + amber-integration Python step): --junit-xml=....
  • bun test (agent-service): --reporter=junit --reporter-outfile=junit.xml.

Test Analytics ingestion is independent of coverage and shares the same CODECOV_TOKEN. Once landed, Codecov will start posting failing-test stack traces on PRs, flagging tests that have gone flaky on main, and tracking per-test runtime / failure-rate over time.

Any related issues, documentation, discussions?

Closes #5206.

How was this PR tested?

Ran sbt scalafmtCheckAll and prettier --check on the two modified vitest.*.config.ts files locally; both pass.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.7

Wire JUnit-XML emission into each stack and add a codecov-action upload
step per job with report_type: test_results so Codecov can surface
failing-test stack traces in PR comments, detect flaky tests against
main, and track per-test runtime/failure-rate over time. Data flows
independently of coverage through the same secrets.CODECOV_TOKEN.
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI ci changes related to CI common labels May 25, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.13%. Comparing base (c435aa7) to head (75048c2).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5207      +/-   ##
============================================
- Coverage     47.15%   47.13%   -0.03%     
+ Complexity     2348     2343       -5     
============================================
  Files          1042     1042              
  Lines         39989    39989              
  Branches       4260     4260              
============================================
- Hits          18855    18847       -8     
- Misses        20012    20015       +3     
- Partials       1122     1127       +5     
Flag Coverage Δ
access-control-service 39.53% <ø> (ø)
agent-service 33.76% <ø> (ø)
amber 50.31% <ø> (-0.06%) ⬇️
computing-unit-managing-service 0.00% <ø> (ø)
config-service 0.00% <ø> (ø)
file-service 32.18% <ø> (ø)
frontend 37.81% <ø> (ø)
python 90.50% <ø> (ø)
workflow-compiling-service 56.81% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Yicong-Huang Yicong-Huang requested a review from aglinxinyuan May 25, 2026 22:45
@Yicong-Huang Yicong-Huang added this pull request to the merge queue May 25, 2026
Merged via the queue into apache:main with commit 9b850e2 May 25, 2026
25 checks passed
@Yicong-Huang Yicong-Huang deleted the ci/codecov-test-analytics branch May 25, 2026 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci changes related to CI common dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable Codecov Test Analytics across all CI test jobs

3 participants