Skip to content

Comments

Add solution filtered parallel test run#4226

Merged
westey-m merged 1 commit intomicrosoft:feature-xunit3-mtp-upgradefrom
westey-m:parallel-tests-run
Feb 24, 2026
Merged

Add solution filtered parallel test run#4226
westey-m merged 1 commit intomicrosoft:feature-xunit3-mtp-upgradefrom
westey-m:parallel-tests-run

Conversation

@westey-m
Copy link
Contributor

Motivation and Context

Description

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings February 24, 2026 21:21
@westey-m westey-m merged commit a39e324 into microsoft:feature-xunit3-mtp-upgrade Feb 24, 2026
4 checks passed
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the .NET test execution from sequential bash-based iteration to parallel execution using Microsoft Testing Platform (MTP). A new PowerShell script filters solution files by target framework, enabling dotnet test --solution to run all tests in parallel across supported projects.

Changes:

  • Added a PowerShell script to dynamically filter .slnx solution files by target framework, removing test projects that don't support the specified framework
  • Replaced bash-based sequential test execution (iterating through projects) with PowerShell-based parallel test execution using filtered solutions and --filter-query
  • Maintained all existing test filtering, coverage collection, and environment configuration while improving performance through parallelization

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
.github/workflows/New-FrameworkFilteredSolution.ps1 New PowerShell script that filters .slnx solution files to include only test projects supporting a specific target framework
.github/workflows/dotnet-build-and-test.yml Migrated unit and integration test execution from bash to PowerShell, using filtered solutions for parallel test execution with --filter-query patterns

# Write the filtered solution
$slnx.Save($OutputPath)

# Report results to stderr so stdout is clean for piping
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on line 98 states "Report results to stderr so stdout is clean for piping", but Write-Host doesn't write to stderr. In PowerShell 5.0 and later, Write-Host writes to the information stream (stream 6), not stderr (stream 2). In earlier versions, it writes directly to the console host.

If the intent is to write to stderr to keep stdout clean, use Write-Error (for errors) or redirect to stderr explicitly using Write-Output "..." | Out-File -FilePath ([Console]::Error) -Append. However, since Write-Host output doesn't interfere with variable assignment or piping (as shown in the examples on lines 30-35), the current implementation works correctly for its intended purpose. The comment should be updated to clarify that Write-Host writes to the information stream, not stderr.

Suggested change
# Report results to stderr so stdout is clean for piping
# Report results via Write-Host (information stream) so stdout is clean for piping

Copilot uses AI. Check for mistakes.
}

# Query the project's target frameworks using MSBuild
$targetFrameworks = & dotnet msbuild $projFullPath -getProperty:TargetFrameworks -p:Configuration=$Configuration -nologo 2>$null
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dotnet msbuild command's stderr output is being redirected to $null (2>$null), which could hide important error messages about project parsing issues. If the project file is malformed or has build errors, these errors would be silently suppressed.

Consider capturing and logging stderr output, or at least preserving warnings while suppressing only expected informational messages. This would help diagnose issues when the script doesn't work as expected.

Suggested change
$targetFrameworks = & dotnet msbuild $projFullPath -getProperty:TargetFrameworks -p:Configuration=$Configuration -nologo 2>$null
$targetFrameworks = & dotnet msbuild $projFullPath -getProperty:TargetFrameworks -p:Configuration=$Configuration -nologo

Copilot uses AI. Check for mistakes.
Comment on lines +30 to +35
$filtered = ./eng/New-FilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472
dotnet test --solution $filtered --no-build -f net472

.EXAMPLE
# Inline usage with dotnet test (PowerShell)
dotnet test --solution (./eng/New-FilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472) --no-build -f net472
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example paths in the script documentation reference "./eng/New-FilteredSolution.ps1", but the script is actually located at ".github/workflows/New-FrameworkFilteredSolution.ps1". This inconsistency between the documentation and the actual file location could confuse users trying to use the script.

Update the example paths in lines 30 and 35 to reflect the actual location: ".github/workflows/New-FrameworkFilteredSolution.ps1".

Suggested change
$filtered = ./eng/New-FilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472
dotnet test --solution $filtered --no-build -f net472
.EXAMPLE
# Inline usage with dotnet test (PowerShell)
dotnet test --solution (./eng/New-FilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472) --no-build -f net472
$filtered = .github/workflows/New-FrameworkFilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472
dotnet test --solution $filtered --no-build -f net472
.EXAMPLE
# Inline usage with dotnet test (PowerShell)
dotnet test --solution (.github/workflows/New-FrameworkFilteredSolution.ps1 -Solution ./agent-framework-dotnet.slnx -TargetFramework net472) --no-build -f net472

Copilot uses AI. Check for mistakes.
Comment on lines +99 to +106
Write-Host "Filtered solution written to: $OutputPath" -ForegroundColor Green
if ($removed.Count -gt 0) {
Write-Host "Removed $($removed.Count) test project(s) not targeting ${TargetFramework}:" -ForegroundColor Yellow
foreach ($r in $removed) {
Write-Host " - $r" -ForegroundColor Yellow
}
}
Write-Host "Kept $($kept.Count) test project(s)." -ForegroundColor Green
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script is called with the -Verbose flag on line 151, which will output verbose messages from the script. However, the script uses Write-Host (lines 99-106) for its main output instead of Write-Verbose. This means the verbose flag won't control whether these messages are displayed - they'll always be shown regardless of the -Verbose setting.

For proper verbose handling, informational messages about which projects are being kept or removed should use Write-Verbose instead of Write-Host, and the current Write-Verbose calls (lines 75, 85, 89) should remain as-is. This would allow the -Verbose flag to properly control the verbosity of the script's output.

Suggested change
Write-Host "Filtered solution written to: $OutputPath" -ForegroundColor Green
if ($removed.Count -gt 0) {
Write-Host "Removed $($removed.Count) test project(s) not targeting ${TargetFramework}:" -ForegroundColor Yellow
foreach ($r in $removed) {
Write-Host " - $r" -ForegroundColor Yellow
}
}
Write-Host "Kept $($kept.Count) test project(s)." -ForegroundColor Green
Write-Verbose "Filtered solution written to: $OutputPath"
if ($removed.Count -gt 0) {
Write-Verbose "Removed $($removed.Count) test project(s) not targeting ${TargetFramework}:"
foreach ($r in $removed) {
Write-Verbose " - $r"
}
}
Write-Verbose "Kept $($kept.Count) test project(s)."

Copilot uses AI. Check for mistakes.
Comment on lines +91 to +96
$proj.ParentNode.RemoveChild($proj) | Out-Null
}
}

# Write the filtered solution
$slnx.Save($OutputPath)
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script sets $ErrorActionPreference = "Stop" on line 51, which will cause the script to terminate on any error. However, when the script removes projects from the XML (line 91), it pipes to Out-Null. If RemoveChild fails for any reason, the script will stop without saving the filtered solution or providing a clear error message about which project caused the issue.

Consider adding try-catch blocks around critical operations (especially the XML manipulation and Save operations) to provide more meaningful error messages and ensure cleanup happens even if errors occur.

Suggested change
$proj.ParentNode.RemoveChild($proj) | Out-Null
}
}
# Write the filtered solution
$slnx.Save($OutputPath)
try {
$proj.ParentNode.RemoveChild($proj) | Out-Null
}
catch {
Write-Error "Failed to remove project '$projRelPath' from solution XML. Error: $($_.Exception.Message)"
}
}
}
# Write the filtered solution
try {
$slnx.Save($OutputPath)
}
catch {
Write-Error "Failed to save filtered solution to '$OutputPath'. Error: $($_.Exception.Message)"
throw
}

Copilot uses AI. Check for mistakes.
$targetFrameworks = & dotnet msbuild $projFullPath -getProperty:TargetFrameworks -p:Configuration=$Configuration -nologo 2>$null
$targetFrameworks = $targetFrameworks.Trim()

if ($targetFrameworks -like "*$TargetFramework*") {
Copy link

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script uses substring matching with the -like operator on line 84 to check if a target framework is supported. While this works correctly for the framework monikers used in this repository (net10.0, net9.0, net8.0, net472), it could cause issues with platform-specific target frameworks in the future.

For example, if a project targets "net8.0-windows" and you filter for "net8.0", the substring match would succeed even though "net8.0" doesn't match "net8.0-windows" exactly. Consider splitting TargetFrameworks by semicolons and checking for exact matches to make the script more robust for future use cases.

Suggested change
if ($targetFrameworks -like "*$TargetFramework*") {
# TargetFrameworks is a semicolon-separated list; match the requested framework exactly
$frameworkList = $targetFrameworks -split ';' | ForEach-Object { $_.Trim() } | Where-Object { $_ -ne '' }
if ($frameworkList -contains $TargetFramework) {

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant