RFC 014 + impl: experimental TestRun.Current / PlannedTests API (#7311)#8461
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an experimental ambient MSTest API (TestRun.Current) for querying run-wide information (initially the filtered/discovered planned tests for the current assembly), and wires it up from the adapter so it’s available during [AssemblyInitialize] and test execution. In addition, the PR introduces a new --ansi terminal option in Microsoft.Testing.Platform to explicitly control ANSI escape emission and precedence over --no-ansi, including updated help text/resources and tests.
Changes:
- Add
[Experimental("MSTESTEXP")]TestRun,ITestRunInfo, andPlannedTestpublic APIs (plus PublicAPI entries) and adapter-side snapshot construction/wiring. - Add
--ansi <auto|on|off>CLI option, argument validator helpers, updated help/info expectations, localized resources, and acceptance/unit tests. - Add adapter unit tests validating
TestRunInfosnapshot creation andTestRun.Currentreset semantics.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestRunInfoTests.cs | New unit coverage for adapter TestRunInfo snapshot behavior and TestRun.Current reset semantics. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/TerminalTestReporterCommandLineOptionsProviderTests.cs | Unit tests for --ansi option validation and option metadata. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineOptionArgumentValidatorTests.cs | Unit tests for boolean/auto argument validation helpers. |
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.cs | Updates MSTest help expectations to include --ansi text. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs | Updates MTP help/info expectations for --ansi option and info output. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs | Updates “all extensions” help/info expectations for --ansi. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/AnsiOptionTests.cs | New acceptance tests validating --ansi behavior (on/off/auto precedence & validation). |
| src/TestFramework/TestFramework.Extensions/TestRun.cs | New ambient TestRun.Current API with internal setter and empty default. |
| src/TestFramework/TestFramework.Extensions/PublicAPI/PublicAPI.Unshipped.txt | Declares new public API surface for shipping validation. |
| src/TestFramework/TestFramework.Extensions/PlannedTest.cs | New public DTO representing a planned test in the current run. |
| src/TestFramework/TestFramework.Extensions/ITestRunInfo.cs | New public run-info interface surfaced via TestRun.Current. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf | Adds new localized resource entries for --ansi strings. |
| src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx | Adds --ansi description and invalid-argument strings. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/TerminalOutputDevice.cs | Implements precedence and mapping from --ansi/--no-ansi to terminal ANSI mode selection. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporterCommandLineOptionsProvider.cs | Registers --ansi option and validates its argument. |
| src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionArgumentValidator.cs | Adds reusable validators for on/off/auto boolean-like option arguments. |
| src/Adapter/MSTestAdapter.PlatformServices/Execution/UnitTestRunner.cs | Wires TestRun.SetCurrent(...) to publish the planned test snapshot before execution. |
| src/Adapter/MSTestAdapter.PlatformServices/Execution/TestRunInfo.cs | Builds ITestRunInfo snapshots from adapter UnitTestElement discovery/execution data. |
| docs/RFCs/014-TestRun-Current-PlannedTests.md | RFC documenting goals, design rationale, and initial surface for TestRun.Current. |
Copilot's findings
- Files reviewed: 31/31 changed files
- Comments generated: 3
|
@copilot resolve the merge conflicts in this pull request |
Resolved by merging |
Adds a static ambient surface `TestRun.Current` exposing
`ITestRunInfo.PlannedTests` so user code (typically
[AssemblyInitialize] or fixtures) can decide whether expensive
setup is needed based on the filtered list of tests that will
actually run in the current assembly.
Surface (all `[Experimental(""MSTESTEXP"")]`):
- `static class TestRun` with `Current`
- `interface ITestRunInfo` with `PlannedTests`
- `sealed class PlannedTest` (sealed class, not record, to
avoid `init` accessors per repo guidelines and to keep
equality semantics off collection fields)
Adapter populates `TestRun.SetCurrent(...)` inside the
`UnitTestRunner` ctor so the snapshot lives in the same
AppDomain/process that runs [AssemblyInitialize] and the
tests themselves (works for both VSTest and MTP paths).
RFC: docs/RFCs/014-TestRun-Current-PlannedTests.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a0caa31 to
56eddce
Compare
- TestRun.Current setter changed from internal to private so SetCurrent(...) is the only mutation path, preserving the never-null invariant. - TestRunInfo.ToPlannedTest now maps TestMethod.DisplayName to null when it equals TestMethod.Name (the default fallback), so consumers can distinguish 'no explicit display name' from 'display name set' as the docs already promise. - Added internal PlannedTest.CreateFromOwnedArrays factory that skips the defensive copy when the adapter already owns freshly-built arrays, avoiding a double allocation for every test with traits. - Added regression test covering the default-display-name-to-null mapping. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds an experimental ambient API for querying information about the current test run.
Closes #7311.
What
Three new
[Experimental("MSTESTEXP")]types inMicrosoft.VisualStudio.TestTools.UnitTesting(shipped viaMSTest.TestFramework.Extensions):static class TestRunwithCurrentreturning anITestRunInfo(never null).interface ITestRunInfoexposingIReadOnlyCollection<PlannedTest> PlannedTests.sealed class PlannedTest— immutable DTO with the metadata needed to decide whether to run expensive setup:FullyQualifiedTestClassName,TestName,TestDisplayName,AssemblyPath,ManagedTypeName,ManagedMethodName,DeclaringFilePath,DeclaringLineNumber,TestCategories,TestProperties.Adapter wiring:
UnitTestRunnerpopulatesTestRun.SetCurrent(...)in its constructor, so the snapshot lives in the same AppDomain/process that runs[AssemblyInitialize]and the tests themselves (works for both the VSTest adapter path and the Microsoft.Testing.Platform path).Why
The motivating use case from #7311: in
[AssemblyInitialize], decide whether to perform expensive partial setup (e.g. build a compatibility solution, spin up a Docker container) based on whether any test matching a given criterion will actually run.Same call works from any helper / fixture / extension, not just lifecycle hooks.
Design notes
Full design rationale in docs/RFCs/014-TestRun-Current-PlannedTests.md (this PR). Highlights:
TestContext.TestContext.Currentis anAsyncLocalthat is null outside test execution, so a static helper / fixture / extension cannot read it; per-run data on a per-test type was also called out as bloat in the issue thread. Matches the two-surface pattern of xUnit v3 and NUnit.PlannedTestis asealed class, not a struct (too many ref-typed fields) and not a record (structural equality across collections is surprising; positional records emitinitaccessors, which the repo guidelines explicitly forbid for new public APIs).FullyQualifiedTestClassName/TestName/TestDisplayNamefromTestContext;DeclaringFilePath/DeclaringLineNumberfromTestMethodAttribute;TestCategoriesfromTestCategoryAttribute;TestPropertiesfrom[TestProperty(Name, Value)].IReadOnlyCollection<string>(noIReadOnlySeton netstandard2.0 / net462),IReadOnlyCollection<KeyValuePair<string, string>>for the multi-valued[TestProperty].TestRun.Currentis never null — defaults to an empty implementation, so callers in any context can just dot through without null checks.Tests
6 new unit tests in
MSTestAdapter.PlatformServices.UnitTests/Execution/TestRunInfoTests.cscovering: empty list, single element happy path, categories + multi-valued properties, non-nullCurrent, reset semantics, and input-collection copy semantics. All adapter unit tests pass. Clean build across net9.0, net8.0, netstandard2.0, net462.Compatibility
Additive only; not a breaking change. All API is
[Experimental("MSTESTEXP")]for v1 so the shape can still evolve before locking based on user feedback.Scope cleanup
The previously bundled
--ansi <auto|on|off>CLI option has been split out into a separate PR: #8493.