Skip to content

Run Wasm.Build.Tests Helix jobs in parallel#130798

Merged
pavelsavara merged 1 commit into
dotnet:mainfrom
pavelsavara:wbt-parallel-helix-sends
Jul 16, 2026
Merged

Run Wasm.Build.Tests Helix jobs in parallel#130798
pavelsavara merged 1 commit into
dotnet:mainfrom
pavelsavara:wbt-parallel-helix-sends

Conversation

@pavelsavara

Copy link
Copy Markdown
Member

Fixes #118938

Problem

Wasm.Build.Tests (WBT, the buildwasmapps scenario) sends its Helix jobs sequentially — each job is sent, then the build blocks on Waiting for completion before the next Sent Helix Job is issued. This serializes the whole WBT run behind the sum of every Helix job's duration.

Root cause

WBT postBuildSteps call eng/pipelines/libraries/helix.yml, which runs src/libraries/sendtohelix.proj. For buildwasmapps, that project fans out into a cartesian product of sub-projects (TestUsingWorkloads × WasmEnableWebcil × WasmFingerprintAssets × bundler-friendly), and invokes them with:

<MSBuild Projects="@(_ProjectsToBuild)" Targets="Test" BuildInParallel="$(_BuildInParallel)" StopOnFirstFailure="false" />

_BuildInParallel is already true when there is more than one sub-project. But BuildInParallel only actually runs builds concurrently when the msbuild process driving it has multiple nodes (/maxcpucount). The Send to Helix step passed no /maxcpucount, so it ran on a single node — each sub-project's Test target (which sends a Helix job and waits for its completion) ran to completion before the next started. That is exactly the sequential Sent Helix JobWaiting for completion pattern in #118938.

Fix

Pass /maxcpucount:8 to the WBT Helix send (via the existing extraHelixArguments), so the already-present BuildInParallel=true sends and awaits the per-combo Helix jobs concurrently. This is the same mechanism eng/pipelines/coreclr/libraries-jitstress.yml already uses (extraHelixArguments: /maxcpucount:10) to parallelize its multi-scenario sends.

Applied to all three WBT senders:

  • eng/pipelines/common/templates/browser-wasm-build-tests.yml (browser Mono)
  • eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml (browser CoreCLR)
  • eng/pipelines/common/templates/simple-wasm-build-tests.yml (Mono single-threaded; used by the extra-platforms wasm pipelines)

Notes

  • The WBT jobs stay in the existing Build stage — no new stage, no dependency on the whole Build stage completing, and no Helix Job Monitor. This is an intentional alternative to Move Wasm.Build.Tests into a dedicated WasmBuildTests pipeline stage #130773 (which moves WBT into a dedicated WasmBuildTests stage); that approach fixes stage organization but delays WBT start until the entire Build stage finishes and does not by itself change the sequential send behavior.
  • StopOnFirstFailure="false" is already set, so parallel sends still aggregate all failures; per-Helix-job timeouts are unchanged.
  • Full effect can only be confirmed by running the pipelines on Azure DevOps; local validation covered YAML parsing and template structure.

Note

This pull request was authored by GitHub Copilot.

WBT (buildwasmapps) fans out sendtohelix.proj into several sendtohelixhelp.proj invocations (Workloads/Webcil/Fingerprinting/Bundler combos). sendtohelix.proj already invokes them with BuildInParallel=true, but the Send to Helix msbuild call passed no /maxcpucount, so it ran on a single node and each Helix job was sent then waited to completion before the next was sent. Pass /maxcpucount:8 (as libraries-jitstress.yml already does) so the per-combo Helix jobs are sent and awaited concurrently within the existing Build stage.
Copilot AI review requested due to automatic review settings July 15, 2026 16:33
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Wasm.Build.Tests (“buildwasmapps”) Helix send steps so sendtohelix.proj can actually execute its existing BuildInParallel=true fan-out concurrently by giving the driving MSBuild process multiple nodes via /maxcpucount:8.

Changes:

  • Add /maxcpucount:8 to the extraHelixArguments passed to eng/pipelines/libraries/helix.yml for the WBT buildwasmapps scenario.
  • Apply the change consistently across the three WBT-sending templates (Mono browser, CoreCLR browser, and “simple” WBT sender).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
eng/pipelines/common/templates/simple-wasm-build-tests.yml Adds /maxcpucount:8 to the Helix send invocation for WBT so multi-combo sends can run concurrently.
eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml Adds /maxcpucount:8 to the CoreCLR WBT Helix send invocation to enable parallel fan-out.
eng/pipelines/common/templates/browser-wasm-build-tests.yml Adds /maxcpucount:8 to the Mono WBT Helix send invocation to enable parallel fan-out.

@pavelsavara pavelsavara added arch-wasm WebAssembly architecture os-browser Browser variant of arch-wasm labels Jul 15, 2026
@pavelsavara pavelsavara added this to the 11.0.0 milestone Jul 15, 2026
@pavelsavara
pavelsavara temporarily deployed to copilot-pat-pool July 15, 2026 20:56 — with GitHub Actions Inactive
@pavelsavara
pavelsavara marked this pull request as ready for review July 15, 2026 20:56
@pavelsavara
pavelsavara requested a review from akoeplinger as a code owner July 15, 2026 20:56
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara
See info in area-owners.md if you want to be subscribed.

@pavelsavara
pavelsavara temporarily deployed to copilot-pat-pool July 15, 2026 20:56 — with GitHub Actions Inactive
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@pavelsavara
pavelsavara merged commit e487c08 into dotnet:main Jul 16, 2026
103 of 106 checks passed
@pavelsavara
pavelsavara deleted the wbt-parallel-helix-sends branch July 16, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch-wasm WebAssembly architecture area-Build-mono os-browser Browser variant of arch-wasm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[browser][WBT] - make WBT test schedule helix jobs in parallel

3 participants