Repository Quality Report: Technical Debt Hygiene (2026-05-19)

[Evangelink]

🎯 Repository Quality Improvement Report — Technical Debt Hygiene

Analysis Date: 2026-05-19
Focus Area: Technical Debt Hygiene (Custom)
Strategy Type: Custom — repository-specific focus area

Executive Summary

Analysis of the microsoft/testfx codebase reveals 69 TODO/FIXME/HACK comments in non-test source files, but only 9 of them (13%) are linked to GitHub issues. The remaining 60 comments represent invisible technical debt — items known to need work but not tracked anywhere that contributors can discover, prioritize, or assign.

Additionally, the codebase has 52 usages of [Obsolete], of which only 11 use error: true. The remaining 41 are soft warnings, which historically see low migration rates. Finally, 43 unshipped public API entries in TestFramework/PublicAPI.Unshipped.txt represent a significant batch of new collection assertion APIs pending their next release, which should be reviewed for consistency and completeness before shipping.

Addressing these issues improves contributor onboarding, prioritization transparency, and API quality discipline across the framework.

Full Analysis Report

Focus Area: Technical Debt Hygiene

Current State Assessment

Metrics Collected:

Metric	Value	Status
TODO/FIXME/HACK comments (non-test source)	69	⚠️
TODOs with linked GitHub issues	9 (13%)	❌
TODOs without issue links	60 (87%)	❌
[Obsolete] usages in src	52	⚠️
[Obsolete] as errors (error: true)	11	✅
[Obsolete] as warnings only	41	⚠️
Pending unshipped public APIs (TestFramework)	43	⚠️
Pending unshipped public APIs (Logging extension)	4	⚠️

Findings

Strengths

• TODOs that do have issue links reference specific actionable items (e.g., #2999, #8086)
• Deprecation errors (error: true) are used correctly for internal cases like HashCode struct misuse and truly unused APIs
• PublicAPI.Unshipped.txt tracking is in place across all packages — the discipline exists

Areas for Improvement

• [HIGH] 60 TODO/FIXME/HACK comments have no GitHub issue link, making them invisible to project planning. Examples:
  • TestNodesGenerator.cs:65 — default namespace resolution needs investigation
  • TestMethodInfo.Execution.cs:366 — // TODO: Avoid blocking. with no issue
  • ExceptionHelper.cs:34 — // TODO:Fix the shadow stack-trace used in Private Object
  • DeploymentItemUtility.cs:130 — potential NRE with no tracking
• [MEDIUM] 41 [Obsolete] usages are warnings only. For a public framework, soft deprecations may not drive migration unless they eventually escalate to errors. A policy for escalation timelines is missing.
• [MEDIUM] 43 new collection assertion APIs in TestFramework/PublicAPI.Unshipped.txt (e.g., AreAllDistinct, AreSequenceEqual, ContainsAll, DoesNotContainAll) represent a large batch of new surface area; a review for naming/parameter consistency before shipping would be valuable.
• [LOW] One HACK comment with no issue link: MSTestAdapterSettings.cs re: AppDomain behavior under VSTest — this category of pragmatic workarounds should be tracked.

---

🤖 Suggested Improvement Tasks

The following actionable tasks address the findings above.

Task 1: Link Untracked TODOs to GitHub Issues

Priority: Medium
Estimated Effort: Medium

60 TODO/FIXME/HACK comments in source files have no associated GitHub issue. This makes them invisible to contributors and project managers. For each comment, either:

• Create a GitHub issue and update the comment to // TODO: <description> (tracked in #NNNN), or
• Delete the comment if the concern is no longer valid

High-priority candidates to triage first:

• src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:366 — blocking async call
• src/Adapter/MSTestAdapter.PlatformServices/Execution/TestExecutionContext.cs:65,69 — test execution state issues
• src/Analyzers/MSTest.SourceGeneration/Generators/TestNodesGenerator.cs:65 — namespace resolution gap
• src/TestFramework/TestFramework/Assertions/Assert.AreAllDistinct.cs:221 — deduplication note with no tracking

---

Task 2: Establish an Obsolete API Escalation Policy

Priority: Medium
Estimated Effort: Small

Currently 41 of 52 [Obsolete] usages are soft warnings. Without a documented escalation timeline (e.g., "warning in vX.Y, error in vX.Y+2"), users have no urgency to migrate and the codebase accumulates legacy surface area indefinitely.

Suggested actions:

• Add a CONTRIBUTING.md or docs/ section documenting the deprecation policy (e.g., minimum 2 minor version warning period before error: true)
• Audit the 41 warning-only obsolete APIs to identify candidates that have been deprecated long enough to escalate to error: true
• Run: grep -rn "\[Obsolete" --include="*.cs" src/ | grep -v "error.*true" to get the full list

---

Task 3: Review and Validate Unshipped Public API Surface

Priority: High
Estimated Effort: Medium

src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt contains 43 pending API entries — a large batch of new collection assertion methods (AreAllDistinct, AreSequenceEqual, ContainsAll, DoesNotContainAll, AreEquivalent, AreNotEquivalent, AreAllNotNull, AreAllOfType). Before these ship:

• Review parameter ordering consistency across overloads (e.g., collection vs expected first)
• Confirm all overloads have matching XML doc comments
• Verify [MSTESTEXP]-tagged APIs (Assert.Scope) are intentionally experimental
• Ensure AssertFailedException.ActualText and ExpectedText new properties have tests covering null/empty scenarios

---

Task 4: Triage High-Risk Untracked Technical Debt Items

Priority: High
Estimated Effort: Small

Several TODO comments describe potential bugs or correctness gaps that should be reviewed immediately:

1. DeploymentItemUtility.cs:130 — // TODO: Check if we can avoid potential NRE here. — potential null reference exception in deployment item handling
2. WinUI_UITestMethodAttribute.cs:46 — // TODO: Code seems to be assuming DeclaringType is never null, but it can be null. — potential NRE in WinUI test execution
3. PrivateObject.cs:724 — // TODO: Check if it's possible to have parameterTypes null here — assertion vs reality mismatch
4. DiagnosticExtensions.cs:140 — // TODO: Revert this reflection based workaround once we move to Microsoft.CodeAnalysis version 3.0 — this workaround may no longer be needed given the current CodeAnalysis version in use

For each: create a GitHub issue or resolve the concern in a PR.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-05-19	Technical Debt Hygiene	Custom

First run — no prior history

---

🎯 Recommendations

Immediate Actions (This Week)

1. Triage the 4 high-risk TODO items (Task 4) — Priority: High
2. Review the 43 unshipped public APIs for consistency before next release (Task 3) — Priority: High

Short-term Actions (This Month)

1. Systematically link the 60 untracked TODOs to GitHub issues (Task 1) — Priority: Medium
2. Document and apply the obsolete API escalation policy (Task 2) — Priority: Medium

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-05-20 — Focus area selected based on diversity algorithm (next: likely standard category or new custom area)

Generated by Repository Quality Improver · ● 3.7M · ◷

• expires on May 21, 2026, 11:09 PM UTC