Skip to content

Parallelize critical CI test workloads - #16116

Draft
jamesfredley wants to merge 4 commits into
8.0.xfrom
perf/8.0.x-ci-wall-clock-regression
Draft

Parallelize critical CI test workloads#16116
jamesfredley wants to merge 4 commits into
8.0.xfrom
perf/8.0.x-ci-wall-clock-regression

Conversation

@jamesfredley

Copy link
Copy Markdown
Contributor

Summary

  • add deterministic SHA-256 sharding for Gradle Test tasks while preserving existing onlyIf predicates
  • split Groovy Joint validation into three shards and Windows JDK 25 main CI into two shards
  • retain one complete build in shard zero and restore dependency-only caches in every shard

No workflow, OS/JDK matrix leg, validation task, or test task is removed.

Verification

  • build-logic: full test suite passes
  • validateActions: all 23 workflows pass
  • Groovy Joint: 401 test-task candidates split 132/129/140 with zero overlap
  • Windows: 401 candidates split 200/201 with zero overlap and no missing baseline tasks
  • Windows sibling graph: 2,811 scheduled tasks versus 6,963 in the current full leg
  • unsharded builds emit no shard manifest and retain existing behavior

Performance Gate

This is a draft measurement PR. The current aggregate pull-request wall-clock baseline is approximately 117 minutes, with Groovy Joint and Windows main CI as the measured limiters. Keep this change only if aggregate completion improves by at least 15% and 10 minutes without reducing coverage.

Introduce a root-only build-logic plugin that partitions Test tasks with
stable SHA-256 assignments while preserving existing execution predicates.
Cover lifecycle validation, late task registration, aggregate test facades,
and disjoint shard manifests with TestKit.

Assisted-by: opencode:gpt-5.6-sol
Split Groovy Joint validation across three deterministic test shards and
the Windows JDK 25 core build across two shards while retaining one complete
build in shard zero. Restore dependency-only caches in every shard and allow
only full-build jobs to save them.

Assisted-by: opencode:gpt-5.6-sol
@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.3497%. Comparing base (a1e526f) to head (7ebf2fb).
⚠️ Report is 19 commits behind head on 8.0.x.

Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                 @@
##                8.0.x     #16116        +/-   ##
==================================================
+ Coverage     52.3431%   52.3497%   +0.0066%     
- Complexity      18296      18302         +6     
==================================================
  Files            2036       2036                
  Lines           96347      96373        +26     
  Branches        16829      16836         +7     
==================================================
+ Hits            50431      50451        +20     
- Misses          38492      38498         +6     
  Partials         7424       7424                

see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jamesfredley

Copy link
Copy Markdown
Contributor Author

First-run measurement

The acceptance gate passes on the unchanged PR SHA.

Critical path Baseline PR #16116 Improvement
Aggregate PR completion 117m24s 86m32s 30m52s / 26.3%
Groovy Joint workflow 117m24s 60m07s 57m17s / 48.8%
Main CI workflow 106m35s 86m32s 20m03s / 18.8%

Shard timings:

  • Groovy Joint: 56m03s, 41m03s, 37m51s after the 3m57s Groovy build
  • Windows JDK 25: 83m55s and 51m00s
  • all sharded jobs passed

The initial Main CI attempt had one unrelated failure in unsharded Spring Security Tests (21): ExtendedSecurityInfoSpec.testLogoutHandlers. The same JDK 25 job passed, the preceding 30 captured JDK 21 runs were green, and the unchanged failed-job rerun passed in 56m51s. No source change was made for that intermittent failure.

The next aggregate limiter is now the unchanged Build Grails-Core Rerunning all Tasks job at 86m27s, narrowly ahead of Windows shard 0.

Expand the Windows core build and rerun-tasks validation to three
deterministic test shards. Keep complete non-test validation and cache writes
on shard zero while every sibling retains the original test and rerun flags.

Assisted-by: opencode:gpt-5.6-sol
@jamesfredley

Copy link
Copy Markdown
Contributor Author

Iteration 2 hosted measurement

All checks passed on cd4490aea5.

Metric Historical baseline Iteration 1 Iteration 2 Total improvement
Aggregate PR wall-clock 117m24s 86m32s 62m09s 55m15s / 47.1%
Main CI workflow 106m35s 86m32s 62m09s 44m26s / 41.7%
Groovy Joint workflow 117m24s 60m07s 26m28s 90m56s / 77.5%

Iteration 2 alone reduced aggregate completion by another 24m23s / 28.2%.

The two former critical jobs moved well below the new floor:

Job Iteration 1 Iteration 2
Rerun all tasks, shard 0 86m27s 46m59s
Windows JDK 25, shard 0 83m55s 33m06s
Windows JDK 25, shard 1 51m00s 27m51s
Windows JDK 25, shard 2 - 30m37s

The critical path is now unchanged functional validation: Functional Tests (Java 21, indy=true) at 62m04s, followed by Java 25 functional tests at 61m46s and Hibernate 7 Java 21 indy tests at 59m52s.

Coverage proof remains exhaustive and disjoint:

  • JDK 21 rerun topology: 401 candidates split 132/129/140, zero overlap.
  • Hosted JDK 25 topology: 416 candidates split 136/132/148, zero overlap.
  • Aggregate report facade remains shard-0-only.

No validation, task, test, platform leg, or rerun flag was removed.

Run: https://github.com/apache/grails-core/actions/runs/31267347078

Split the longest functional and Hibernate 7 validation legs across two
deterministic test shards while preserving every Java and indy configuration.
Keep full non-test verification on the original matrix entries.

Assisted-by: opencode:gpt-5.6-sol
@testlens-app

testlens-app Bot commented Aug 8, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: 7ebf2fb
▶️ Tests: 46925 executed
🟡 Checks: 59/66 completed


Learn more about TestLens at testlens.app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant