Repository Quality: Public API Surface Versioning Hygiene and Experimental Graduation Readiness

[Evangelink]

🎯 Repository Quality Improvement Report — Public API Surface Versioning Hygiene

Analysis Date: 2026-06-22 Focus Area: Custom — Public API Surface Versioning Hygiene Strategy: Custom (60% bucket)

Executive Summary

Analysis of 33 PublicAPI.Shipped/Unshipped.txt pairs found 118 total unshipped API entries queued for the next release (83 in TestFramework, 16 in TestFramework.Extensions, 11 in Platform extensions, 3 in PlatformServices). Three hygiene issues emerged: (1) inconsistent [TPEXP] markers on the four new extension packages; (2) long-lived shipped [MSTESTEXP] experimental types with no documented graduation timeline; (3) new complex experimental API groups lacking explicit stabilization criteria.

The most immediately actionable finding is a plain mis-annotation: TestingPlatformBuilderHook in Microsoft.Testing.Extensions.CtrfReport is [TPEXP] — contradicting the stable-hook pattern used by every other extension package (TrxReport, Retry, CrashDump, HangDump, HotReload, MSBuild, AzureFoundry). This is a one-line fix.

Full Analysis

Metrics

Metric	Value	Status
PublicAPI file pairs	33	✅
Total unshipped entries	118	⚠️ Large batch
[MSTESTEXP] in unshipped	19	⚠️
[TPEXP] in unshipped	11	⚠️
[MSTESTEXP] in shipped (long-lived)	8 (RetryContext/RetryResult)	⚠️
init accessor violations in unshipped	0	✅
Duplicate shipped+unshipped entries	0	✅
Extension packages with marker inconsistency	3	❌

Strengths

• No init accessor violations in new unshipped APIs — code standards are respected.
• No duplicate entries between shipped and unshipped files.
• TestArchitectures.RiscV64 (#if NET9_0_OR_GREATER) correctly placed in net9.0/PublicAPI.Unshipped.txt.
• #nullable enable header present in every file.
• Clear stable-hook pattern established across eight shipped extension packages.

Issues Found

[HIGH] CtrfReport TestingPlatformBuilderHook incorrectly marked [TPEXP]
File: src/Platform/Microsoft.Testing.Extensions.CtrfReport/PublicAPI/PublicAPI.Unshipped.txt
Every other extension (TrxReport, Retry, CrashDump, HangDump, HotReload, MSBuild, AzureFoundry, HtmlReport, JUnitReport) ships its hook as stable. CtrfReport's hook with [TPEXP] is a mis-annotation that will surface a compiler warning to composition consumers.

[MEDIUM] HtmlReport/JUnitReport/CtrfReport extension methods are [TPEXP]
Established stable packages (TrxReport, Retry, CrashDump, HangDump, HotReload, MSBuild, Telemetry) ship Add*Provider extension methods as stable. The three new packages mark theirs as [TPEXP]. If intentional (preview launch), each needs a tracking issue with graduation criteria; if unintentional, the markers should be removed.

[MEDIUM] RetryContext/RetryResult are long-lived [MSTESTEXP] in shipped with no graduation timeline
8 entries for these types have been in PublicAPI.Shipped.txt as experimental with no documented stabilization criteria. Consumers must suppress MSTESTEXP warnings indefinitely.

[MEDIUM] New Assert and TestRunInfo experimental APIs lack graduation criteria

• Unshipped [MSTESTEXP]: Assert.AddValueFormatter, Assert.AddValueFormatter<T>, Assert.Scope() (3 entries)
• Unshipped [MSTESTEXP]: ITestRunInfo, PlannedTest (14 members), TestRun.Current (16 entries in Extensions)

[LOW] ITestRunInfo/PlannedTest/TestRun.Current TFM scope unclear
All 16 entries are in the top-level (all-TFM) TestFramework.Extensions/PublicAPI/PublicAPI.Unshipped.txt, implying availability on net462, netstandard2.0, UWP, and WinUI TFMs. Verify no net8.0+ dependency exists; if one does, entries should move to the net/ subdirectory.

---

🤖 Suggested Improvement Tasks

Task 1: Fix CtrfReport TestingPlatformBuilderHook experimental marker

Priority: High | Effort: Small (1-line change)

In src/Platform/Microsoft.Testing.Extensions.CtrfReport/PublicAPI/PublicAPI.Unshipped.txt, remove [TPEXP] from the two hook lines:

-[TPEXP]Microsoft.Testing.Extensions.CtrfReport.TestingPlatformBuilderHook
-[TPEXP]static Microsoft.Testing.Extensions.CtrfReport.TestingPlatformBuilderHook.AddExtensions(...) -> void
+Microsoft.Testing.Extensions.CtrfReport.TestingPlatformBuilderHook
+static Microsoft.Testing.Extensions.CtrfReport.TestingPlatformBuilderHook.AddExtensions(...) -> void

Also remove the [Experimental] attribute (if present) from the TestingPlatformBuilderHook class in the corresponding source file.

---

Task 2: Decide and document graduation plan for new report extension APIs

Priority: Medium | Effort: Small (decision + tracking issue)

For HtmlReportExtensions, JUnitReportExtensions, and CtrfReportExtensions:

• If intentionally experimental (preview launch): create one tracking issue per package with graduation criteria and add TODO(#NNNN) comments referencing it.
• If unintentional: remove [TPEXP] markers and remove [Experimental] attributes from the source classes.

---

Task 3: Create graduation criteria for RetryContext and RetryResult

Priority: Medium | Effort: Small

Create a tracking issue documenting which open design questions or feedback must be resolved before RetryContext and RetryResult graduate from [MSTESTEXP], and the target version. Add TODO(#NNNN) comments in RetryContext.cs and RetryResult.cs.

---

Task 4: Document stabilization criteria for Assert.Scope and value formatter APIs

Priority: Medium | Effort: Small

Create a tracking issue or milestone note with acceptance criteria for Assert.Scope(), Assert.AddValueFormatter, and Assert.AddValueFormatter<T> — e.g., whether the IDisposable scope semantics are finalized, whether the factory signature is stable. Add TODO(#NNNN) comments in the implementation.

---

Task 5: Verify TFM scope of ITestRunInfo/PlannedTest/TestRun.Current

Priority: Low | Effort: Small

Check whether the implementation of TestRun.Current, ITestRunInfo, and PlannedTest (in TestFramework.Extensions) has any net8.0+ dependencies. If it does, move the 16 unshipped entries from the top-level PublicAPI.Unshipped.txt to the net/PublicAPI.Unshipped.txt subdirectory so the public-API analyzer enforces the correct TFM boundary.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-05-22	test-framework-api-ergonomics	Custom
2026-06-08	todo-comment-policy-compliance	Custom
2026-06-09	mstest-diagnostic-id-governance	Custom
2026-06-10	analyzer-code-fix-coverage-gap	Custom
2026-06-11	public-api-xml-documentation	Standard
2026-06-12	dependency-health	Standard
2026-06-15	experimental-api-lifecycle-governance	Custom
2026-06-16	target-framework-support-matrix	Custom
2026-06-17	localization-pipeline-health	Custom
2026-06-18	banned-symbols-assertion-enforcement	Custom
2026-06-19	legacy-assertion-structured-message-gap	Custom

Next analysis: 2026-06-23 — Focus area selected via diversity algorithm

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Repository Quality Improver workflow. · 587.1 AIC · ⌖ 25.5 AIC · ⊞ 45.6K · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jun 24, 2026, 10:57 PM UTC