Skip to content

Reduce Helix work items with platform-aware partitioning#54502

Merged
MichaelSimons merged 1 commit into
mainfrom
michaelsimons/musical-lamp
May 29, 2026
Merged

Reduce Helix work items with platform-aware partitioning#54502
MichaelSimons merged 1 commit into
mainfrom
michaelsimons/musical-lamp

Conversation

@MichaelSimons
Copy link
Copy Markdown
Member

Summary

Consolidate Helix work item partitions to target ~5 minutes per work item, reducing overall work item count by ~58%.

Problem

The current base method limit of 16 methods per shard was producing far too many short-running work items, especially on faster platforms like mac-arm64 where the average work item took only 42 seconds.

Changes

All partitioning configuration now lives in test/UnitTests.proj:

  • Raised base method limit from 16 → 32 (64 for FullFramework, 128 for osx-arm64)
  • Moved all limit configuration into UnitTests.proj — the C# task (SDKCustomCreateXUnitWorkItemsWithTestExclusion) is now a consumer of BaseMethodLimit rather than hardcoding values
  • Formula: effective limit = BaseMethodLimit × MethodLimitMultiplier

Projected Impact

Queue Current Avg Projected Avg Work Items
Windows 2.9 min ~5.7 min 590 → ~295
mac-x64 2.2 min ~4.5 min 352 → ~176
Linux 1.9 min ~3.8 min 347 → ~174
mac-arm64 0.7 min ~5.6 min 347 → ~44
Total 1,636 → ~689 (~58% reduction)

Configuration at a glance (test/UnitTests.proj)

<!-- Base method limit: platform-aware -->
<BaseMethodLimit Condition="'$(TestFullMSBuild)' == 'true'">64</BaseMethodLimit>
<BaseMethodLimit Condition="$(TargetRid.StartsWith('osx-arm64'))">128</BaseMethodLimit>
<BaseMethodLimit Condition="'$(BaseMethodLimit)' == ''">32</BaseMethodLimit>

<!-- Per-project multipliers -->
<!-- NetAnalyzers:      5x  (effective: 160 / 640 on arm64) -->
<!-- TemplateEngine:   20x  (effective: 640 - prevents splitting) -->
<!-- ApiCompat/GenAPI: 10x  (effective: 320 - prevents splitting) -->

…re base method limit

Move all partitioning configuration into UnitTests.proj:
- Introduce BaseMethodLimit property (32 default, 64 FullFramework, 128 osx-arm64)
- Task now consumes BaseMethodLimit instead of hardcoding values
- Base raised from 16 to 32 for all platforms to target ~5 min per work item
- osx-arm64 gets 128 (4x) since it runs ~4x faster than Windows

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 29, 2026 14:57
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes Helix xUnit work-item partition sizing in test/UnitTests.proj and makes partitioning platform-aware to reduce short-running Helix shards.

Changes:

  • Adds BaseMethodLimit configuration with larger defaults for FullFramework and osx-arm64 legs.
  • Passes BaseMethodLimit into the custom xUnit Helix work-item task.
  • Updates the task to consume the configured base limit and apply per-project multipliers.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
test/UnitTests.proj Defines platform-aware base method limits for Helix test partitioning.
test/xunit-runner/XUnitRunner.targets Passes the configured base method limit into the Helix work-item creation task.
test/HelixTasks/SDKCustomCreateXUnitWorkItemsWithTestExclusion.cs Uses BaseMethodLimit instead of hardcoded task-local defaults when scheduling test shards.

Copy link
Copy Markdown
Member

@chcosta chcosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

musical-lamp?

@MichaelSimons
Copy link
Copy Markdown
Member Author

musical-lamp?

Compliments of the github copilot app!

@MichaelSimons
Copy link
Copy Markdown
Member Author

Mac x64 hung but the rest of the affect legs completed. There is some additional fine tuning to do but I am of the mindset to get this in asap to get some immediate gains and see the impact to the overall system.

Here are the actual workitem impact of this PR. The PR descriptions was theoretical in the sense that test assemblies are the granularity limit.

image

Force merging ahead of a full PR validation run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants