Skip to content

Use partial evaluation in dotnet test setup - #55633

Merged
Evangelink merged 3 commits into
mainfrom
dev/amauryleve/optimize-mtp-evaluation
Aug 6, 2026
Merged

Use partial evaluation in dotnet test setup#55633
Evangelink merged 3 commits into
mainfrom
dev/amauryleve/optimize-mtp-evaluation

Conversation

@Evangelink

@Evangelink Evangelink commented Aug 6, 2026

Copy link
Copy Markdown
Member

Summary

Use MSBuild partial evaluation in two dotnet test setup paths that consume only early evaluation results:

  • stop after properties when device handling only needs TargetFramework and TargetFrameworks
  • stop after items when the pre-build environment-variable path checks ProjectCapability and IntermediateOutputPath

Core project/module discovery continues to fully evaluate each project and target framework exactly once. MTP applications execute DeployToDevice and ComputeRunArguments, so their ProjectInstance must contain registered targets; partial ProjectInstance evaluation is not resumable and must not be followed by a second full evaluation.

The existing shared EvaluationContext remains scoped across each post-build discovery operation to reuse SDK resolution and filesystem observations between its full project evaluations.

Evaluation lifetime

The pre-build device-selection evaluation and post-build discovery intentionally use different shared evaluation contexts. This prevents filesystem observations cached before the build from hiding imports or generated files created by the build.

The regression test creates a conditional props import during the build and verifies that post-build discovery observes it. It also removes the generated import before each invocation so repeated local runs continue to exercise the cache boundary.

Tests

  • ItUsesFreshEvaluationContextAfterBuild
  • focused dotnet.Tests coverage for MTP multi-target discovery, traversal/non-test filtering, VSTest non-test handling, and device/environment target execution

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e05d5996-f5a3-444d-bd9c-1bd2a661b3ef
Copilot AI review requested due to automatic review settings August 6, 2026 11:30
@Evangelink
Evangelink requested a review from a team as a code owner August 6, 2026 11:30
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates dotnet test’s Microsoft.Testing.Platform (MTP) project discovery to use MSBuild partial evaluation (stopping after Properties/Items where possible) to reduce discovery-time evaluation work while preserving correctness where full evaluation is required.

Changes:

  • Introduces ProjectEvaluationStage-based partial evaluation in MTP discovery, with full evaluation reserved for MTP applications that need target execution.
  • Uses partial evaluation when inspecting capabilities/items during build/restore and when selecting a target framework for device scenarios.
  • Adds a regression test asset + test to validate post-build discovery observes newly generated imports (fresh evaluation context after build).
Show a summary per file
File Description
test/TestAssets/TestProjects/DotnetTestDevices/DotnetTestDevices.csproj Adds a conditional post-build props import and a build-time generator target for the regression scenario.
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestSelectsDevice.cs Adds a test covering the fresh evaluation context boundary after build.
src/Cli/dotnet/Commands/Test/MTP/SolutionAndProjectUtility.cs Switches project discovery evaluations to partial stages and re-evaluates to Full when MTP target execution is needed.
src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs Uses Items-stage evaluation when probing runtime-environment-variable capability during build/restore.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs Uses Properties-stage evaluation when reading TargetFramework(s) for device-driven framework selection.

Copilot's findings

  • Files reviewed: 5/5 changed files
  • Comments generated: 1

Comment thread test/TestAssets/TestProjects/DotnetTestDevices/DotnetTestDevices.csproj Outdated
@Evangelink
Evangelink enabled auto-merge August 6, 2026 11:51
@Evangelink

Copy link
Copy Markdown
Member Author

@ViktorHofer @baronfel I have tried to adopt the partial eval features to dotnet test, please review.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e05d5996-f5a3-444d-bd9c-1bd2a661b3ef
Comment thread src/Cli/dotnet/Commands/Test/MTP/SolutionAndProjectUtility.cs Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e05d5996-f5a3-444d-bd9c-1bd2a661b3ef
@Evangelink Evangelink changed the title Use partial evaluation in dotnet test discovery Use partial evaluation in dotnet test setup Aug 6, 2026
@Evangelink
Evangelink merged commit 6d5079c into main Aug 6, 2026
24 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/optimize-mtp-evaluation branch August 6, 2026 14:15
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-rc1 milestone Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants