Skip to content

RFC 014 + impl: experimental TestRun.Current / PlannedTests API (#7311)#8461

Merged
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/test-run-current-planned-tests
May 22, 2026
Merged

RFC 014 + impl: experimental TestRun.Current / PlannedTests API (#7311)#8461
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/test-run-current-planned-tests

Conversation

@Evangelink
Copy link
Copy Markdown
Member

@Evangelink Evangelink commented May 21, 2026

Adds an experimental ambient API for querying information about the current test run.

Closes #7311.

What

Three new [Experimental("MSTESTEXP")] types in Microsoft.VisualStudio.TestTools.UnitTesting (shipped via MSTest.TestFramework.Extensions):

  • static class TestRun with Current returning an ITestRunInfo (never null).
  • interface ITestRunInfo exposing IReadOnlyCollection<PlannedTest> PlannedTests.
  • sealed class PlannedTest — immutable DTO with the metadata needed to decide whether to run expensive setup: FullyQualifiedTestClassName, TestName, TestDisplayName, AssemblyPath, ManagedTypeName, ManagedMethodName, DeclaringFilePath, DeclaringLineNumber, TestCategories, TestProperties.

Adapter wiring: UnitTestRunner populates TestRun.SetCurrent(...) in its constructor, so the snapshot lives in the same AppDomain/process that runs [AssemblyInitialize] and the tests themselves (works for both the VSTest adapter path and the Microsoft.Testing.Platform path).

Why

The motivating use case from #7311: in [AssemblyInitialize], decide whether to perform expensive partial setup (e.g. build a compatibility solution, spin up a Docker container) based on whether any test matching a given criterion will actually run.

[AssemblyInitialize]
public static void Init(TestContext _)
{
    if (TestRun.Current.PlannedTests.Any(t => t.TestCategories.Contains("Compatibility")))
    {
        BuildCompatibilitySolution();
    }
}

Same call works from any helper / fixture / extension, not just lifecycle hooks.

Design notes

Full design rationale in docs/RFCs/014-TestRun-Current-PlannedTests.md (this PR). Highlights:

  • Separate static surface, not on TestContext. TestContext.Current is an AsyncLocal that is null outside test execution, so a static helper / fixture / extension cannot read it; per-run data on a per-test type was also called out as bloat in the issue thread. Matches the two-surface pattern of xUnit v3 and NUnit.
  • PlannedTest is a sealed class, not a struct (too many ref-typed fields) and not a record (structural equality across collections is surprising; positional records emit init accessors, which the repo guidelines explicitly forbid for new public APIs).
  • Naming mirrors existing MSTest surface: FullyQualifiedTestClassName / TestName / TestDisplayName from TestContext; DeclaringFilePath / DeclaringLineNumber from TestMethodAttribute; TestCategories from TestCategoryAttribute; TestProperties from [TestProperty(Name, Value)].
  • Collection types stay at the minimum that meets the use case: IReadOnlyCollection<string> (no IReadOnlySet on netstandard2.0 / net462), IReadOnlyCollection<KeyValuePair<string, string>> for the multi-valued [TestProperty].
  • TestRun.Current is never null — defaults to an empty implementation, so callers in any context can just dot through without null checks.

Tests

6 new unit tests in MSTestAdapter.PlatformServices.UnitTests/Execution/TestRunInfoTests.cs covering: empty list, single element happy path, categories + multi-valued properties, non-null Current, reset semantics, and input-collection copy semantics. All adapter unit tests pass. Clean build across net9.0, net8.0, netstandard2.0, net462.

Compatibility

Additive only; not a breaking change. All API is [Experimental("MSTESTEXP")] for v1 so the shape can still evolve before locking based on user feedback.

Scope cleanup

The previously bundled --ansi <auto|on|off> CLI option has been split out into a separate PR: #8493.

Copilot AI review requested due to automatic review settings May 21, 2026 11:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an experimental ambient MSTest API (TestRun.Current) for querying run-wide information (initially the filtered/discovered planned tests for the current assembly), and wires it up from the adapter so it’s available during [AssemblyInitialize] and test execution. In addition, the PR introduces a new --ansi terminal option in Microsoft.Testing.Platform to explicitly control ANSI escape emission and precedence over --no-ansi, including updated help text/resources and tests.

Changes:

  • Add [Experimental("MSTESTEXP")] TestRun, ITestRunInfo, and PlannedTest public APIs (plus PublicAPI entries) and adapter-side snapshot construction/wiring.
  • Add --ansi <auto|on|off> CLI option, argument validator helpers, updated help/info expectations, localized resources, and acceptance/unit tests.
  • Add adapter unit tests validating TestRunInfo snapshot creation and TestRun.Current reset semantics.
Show a summary per file
File Description
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestRunInfoTests.cs New unit coverage for adapter TestRunInfo snapshot behavior and TestRun.Current reset semantics.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/TerminalTestReporterCommandLineOptionsProviderTests.cs Unit tests for --ansi option validation and option metadata.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineOptionArgumentValidatorTests.cs Unit tests for boolean/auto argument validation helpers.
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.cs Updates MSTest help expectations to include --ansi text.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs Updates MTP help/info expectations for --ansi option and info output.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs Updates “all extensions” help/info expectations for --ansi.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/AnsiOptionTests.cs New acceptance tests validating --ansi behavior (on/off/auto precedence & validation).
src/TestFramework/TestFramework.Extensions/TestRun.cs New ambient TestRun.Current API with internal setter and empty default.
src/TestFramework/TestFramework.Extensions/PublicAPI/PublicAPI.Unshipped.txt Declares new public API surface for shipping validation.
src/TestFramework/TestFramework.Extensions/PlannedTest.cs New public DTO representing a planned test in the current run.
src/TestFramework/TestFramework.Extensions/ITestRunInfo.cs New public run-info interface surfaced via TestRun.Current.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf Adds new localized resource entries for --ansi strings.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx Adds --ansi description and invalid-argument strings.
src/Platform/Microsoft.Testing.Platform/OutputDevice/TerminalOutputDevice.cs Implements precedence and mapping from --ansi/--no-ansi to terminal ANSI mode selection.
src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporterCommandLineOptionsProvider.cs Registers --ansi option and validates its argument.
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionArgumentValidator.cs Adds reusable validators for on/off/auto boolean-like option arguments.
src/Adapter/MSTestAdapter.PlatformServices/Execution/UnitTestRunner.cs Wires TestRun.SetCurrent(...) to publish the planned test snapshot before execution.
src/Adapter/MSTestAdapter.PlatformServices/Execution/TestRunInfo.cs Builds ITestRunInfo snapshots from adapter UnitTestElement discovery/execution data.
docs/RFCs/014-TestRun-Current-PlannedTests.md RFC documenting goals, design rationale, and initial surface for TestRun.Current.

Copilot's findings

  • Files reviewed: 31/31 changed files
  • Comments generated: 3

Comment thread src/Platform/Microsoft.Testing.Platform/OutputDevice/TerminalOutputDevice.cs Outdated
Comment thread src/TestFramework/TestFramework.Extensions/PlannedTest.cs
@Evangelink
Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

Copilot AI review requested due to automatic review settings May 22, 2026 09:09
@Evangelink Evangelink review requested due to automatic review settings May 22, 2026 09:09
Copy link
Copy Markdown
Contributor

Copilot AI commented May 22, 2026

@copilot resolve the merge conflicts in this pull request

Resolved by merging origin/main into this branch and fixing the conflict in TerminalOutputDevice.cs in commit a0caa3163.

Adds a static ambient surface `TestRun.Current` exposing
`ITestRunInfo.PlannedTests` so user code (typically
[AssemblyInitialize] or fixtures) can decide whether expensive
setup is needed based on the filtered list of tests that will
actually run in the current assembly.

Surface (all `[Experimental(""MSTESTEXP"")]`):
  - `static class TestRun` with `Current`
  - `interface ITestRunInfo` with `PlannedTests`
  - `sealed class PlannedTest` (sealed class, not record, to
    avoid `init` accessors per repo guidelines and to keep
    equality semantics off collection fields)

Adapter populates `TestRun.SetCurrent(...)` inside the
`UnitTestRunner` ctor so the snapshot lives in the same
AppDomain/process that runs [AssemblyInitialize] and the
tests themselves (works for both VSTest and MTP paths).

RFC: docs/RFCs/014-TestRun-Current-PlannedTests.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 22, 2026 11:20
@Evangelink Evangelink force-pushed the dev/amauryleve/test-run-current-planned-tests branch from a0caa31 to 56eddce Compare May 22, 2026 11:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 8/8 changed files
  • Comments generated: 4

Comment thread src/TestFramework/TestFramework.Extensions/TestRun.cs
Comment thread src/TestFramework/TestFramework.Extensions/PlannedTest.cs
Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TestRunInfo.cs Outdated
Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TestRunInfo.cs
- TestRun.Current setter changed from internal to private so SetCurrent(...) is the only mutation path, preserving the never-null invariant.
- TestRunInfo.ToPlannedTest now maps TestMethod.DisplayName to null when it equals TestMethod.Name (the default fallback), so consumers can distinguish 'no explicit display name' from 'display name set' as the docs already promise.
- Added internal PlannedTest.CreateFromOwnedArrays factory that skips the defensive copy when the adapter already owns freshly-built arrays, avoiding a double allocation for every test with traits.
- Added regression test covering the default-display-name-to-null mapping.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink merged commit b625999 into main May 22, 2026
74 of 78 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/test-run-current-planned-tests branch May 22, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Knowing which tests will run in AssemblyInitialize

3 participants