Skip to content

🎯 Repository Quality Improvement β€” Exception Handling and Error Recovery PatternsΒ #8479

@Evangelink

Description

@Evangelink

🎯 Repository Quality Improvement Report β€” Exception Handling and Error Recovery Patterns

Analysis Date: 2026-05-21
Focus Area: Exception Handling and Error Recovery Patterns
Strategy Type: Custom (Reuse-inspired deep dive)

Executive Summary

This analysis examined exception handling patterns across the microsoft/testfx repository (2,919 C# source files). The repository demonstrates strong exception handling foundations with 1,249 ConfigureAwait usages, 82 exception filters, and 211 nameof usages in exceptions. However, several modernization opportunities and consistency gaps exist:

Key Findings:

  • Zero adoption of ArgumentNullException.ThrowIfNull (.NET 6+ helper) despite 181 ArgumentNullException throws
  • 5 instances of throw ex; anti-pattern that lose stack traces (in ClassCleanupManager.cs)
  • 50 async void methods creating unhandled exception risk vectors
  • 199 hardcoded exception messages vs 121 localized ones (62% hardcoded rate)
  • Minimal custom exception infrastructure: Only 9 custom exception types, 3 missing standard constructors

The codebase excels at async exception safety (1,249 ConfigureAwait calls) and sophisticated exception filtering (82 when clauses). These strengths can be leveraged to address the identified gaps through targeted improvements.

Full Analysis Report

Focus Area: Exception Handling and Error Recovery Patterns

Current State Assessment

Metrics Collected:

Metric Value Status
Total C# source files 2,919 πŸ“Š
Files with catch blocks 134 βœ…
Files with throw statements 887 βœ…
Top exception thrown InvalidOperationException (266) βœ…
Custom exception types 9 ⚠️
ConfigureAwait usage 1,249 βœ…
Exception filters (when clauses) 82 βœ…
ArgumentNullException.ThrowIfNull adoption 0 ❌
throw ex; anti-patterns 5 ⚠️
Async void methods 50 ⚠️
Exception documentation tags 159 ⚠️
Hardcoded exception messages 199 ⚠️
Localized exception messages 121 βœ…
Bare throw; (correct pattern) 32 βœ…
ExceptionDispatchInfo usage 9 βœ…
AggregateException handling 42 βœ…

Exception Type Distribution

Top 10 Exception Types Thrown:

  1. InvalidOperationException: 266
  2. ArgumentNullException: 181
  3. NotImplementedException: 171
  4. ArgumentException: 72
  5. NotSupportedException: 48
  6. ArgumentOutOfRangeException: 26
  7. PlatformNotSupportedException: 19
  8. TypeInspectionException: 17 (custom)
  9. AssertFailedException: 16 (custom)
  10. FormatException: 15

Custom Exception Types:

  • UnitTestAssertException (abstract base) β€” public API, properly serializable
  • AssertFailedException β€” two versions (TestFramework & MSTest.Engine)
  • AssertInconclusiveException β€” TestFramework assertion outcome
  • MSTestException β€” internal TestFramework exception
  • AdapterSettingsException β€” internal adapter configuration errors
  • TypeInspectionException β€” internal reflection failures
  • TestFailedException β€” internal test execution failures
  • InvalidManagedNameException β€” internal name parsing failures
  • MessageFormatException β€” internal message formatting errors

Findings

Strengths

  1. Excellent Async Exception Safety βœ…

    • 1,249 ConfigureAwait(false) calls across 487 async methods
    • Consistent pattern in analyzers, code fixes, and platform services
    • Prevents deadlocks and context-switching issues
  2. Sophisticated Exception Filtering βœ…

    • 82 exception filters using catch (...) when (...) clauses
    • Excellent example in Assert.AreEquivalent.Comparer.cs:
      catch (Exception ex) when (ex is not OutOfMemoryException 
          and not StackOverflowException 
          and not UnitTestAssertException)
    • Prevents catching non-recoverable exceptions (OutOfMemoryException, StackOverflowException)
  3. Strong nameof Usage βœ…

    • 211 exceptions use nameof for parameter names
    • Refactoring-safe, compile-time checked parameter references
  4. Advanced Exception Preservation βœ…

    • 9 ExceptionDispatchInfo usages for preserving stack traces
    • AssertScope.cs properly aggregates multiple assertion failures
    • Correct use of bare throw; (32 instances) vs throw ex; (only 5)
  5. Proper AggregateException Handling βœ…

    • 42 AggregateException usages, primarily in TestResult.cs and AssertScope.cs
    • Correctly aggregates multiple assertion failures for scope-based assertions

Areas for Improvement

  1. Zero Modern Null Validation Adoption ❌ HIGH PRIORITY

    • 181 ArgumentNullException throws but 0 uses of ArgumentNullException.ThrowIfNull
    • .NET 6+ provides ArgumentNullException.ThrowIfNull(arg) helper
    • Repository targets .NET 8, should leverage modern patterns
    • Current pattern requires manual null checks + throw statements
  2. Stack Trace Loss Anti-Pattern ⚠️ HIGH PRIORITY

    • 5 instances of throw ex; in production code:
      • src/Adapter/MSTestAdapter.PlatformServices/Execution/ClassCleanupManager.cs:70
      • src/Adapter/MSTestAdapter.PlatformServices/Execution/ClassCleanupManager.cs:86
    • Should use bare throw; to preserve stack traces
    • Affects debugging and error diagnosis
  3. Async Void Methods (Unhandled Exception Risk) ⚠️ MEDIUM PRIORITY

    • 50 async void methods in codebase
    • Unhandled exceptions in async void methods crash the process
    • Most are in test code (with VSTHRD100 suppressions), but requires audit
    • Best practice: async Task methods with proper exception handling
  4. Incomplete Custom Exception Design ⚠️ MEDIUM PRIORITY

    • AdapterSettingsException.cs: Missing parameterless constructor, innerException constructor
    • Several custom exceptions lack standard constructor overloads
    • Pattern inconsistency: TypeInspectionException has full constructors, others don't
    • Impacts exception propagation and serialization scenarios
  5. Hardcoded Exception Messages ⚠️ MEDIUM PRIORITY

    • 199 hardcoded vs 121 localized exception messages (62% hardcoded)
    • Violates localization guidelines for user-facing errors
    • Examples found across test infrastructure and platform services
    • Should use resource strings for consistent localization
  6. Limited Exception Documentation ⚠️ LOW PRIORITY

    • Only 159 <exception> XML documentation tags across 2,919 files
    • Public API methods often lack exception documentation
    • Makes it harder for consumers to understand error conditions
  7. Exception-Related Technical Debt ⚠️ LOW PRIORITY


πŸ€– Suggested Improvement Tasks

The following actionable tasks address the findings above, prioritized by impact and effort.

Task 1: Adopt ArgumentNullException.ThrowIfNull for Null Validation

Priority: High
Estimated Effort: Medium
Impact: Modernization, code consistency, reduced boilerplate

Description:

Replace traditional null-check patterns with the modern .NET 6+ ArgumentNullException.ThrowIfNull helper across the codebase.

Current Pattern (181 occurrences):

if (parameter is null)
{
    throw new ArgumentNullException(nameof(parameter));
}

Modern Pattern:

ArgumentNullException.ThrowIfNull(parameter);

Benefits:

  • Single-line null validation
  • Consistent with modern .NET idioms
  • Reduces boilerplate by ~3 lines per validation
  • Compile-time safety with nameof embedded in helper

Target Files: All files throwing ArgumentNullException (181 instances found)

Acceptance Criteria:

  • Replace at least 80% of traditional null checks with ThrowIfNull
  • Update coding guidelines to mandate ThrowIfNull for new code
  • Add analyzer rule (if not present) to suggest ThrowIfNull

Task 2: Fix Stack Trace Loss Anti-Pattern in ClassCleanupManager

Priority: High
Estimated Effort: Small
Impact: Debugging, production diagnostics

Description:

Fix the 5 instances of throw ex; anti-pattern that lose original stack traces. This primarily affects ClassCleanupManager.cs which handles test class cleanup exceptions.

Affected Files:

  • src/Adapter/MSTestAdapter.PlatformServices/Execution/ClassCleanupManager.cs (lines 70, 86)

Current Anti-Pattern:

catch (Exception ex)
{
    // ... processing ...
    throw ex; // ❌ Loses original stack trace
}

Correct Pattern:

catch (Exception ex)
{
    // ... processing ...
    throw; // βœ… Preserves original stack trace
}

Impact: When test cleanup methods throw exceptions, stack traces currently reset to the rethrow point, obscuring the original failure location.

Acceptance Criteria:

  • Replace all 5 throw ex; instances with bare throw;
  • Verify stack traces preserve original exception location in test scenarios
  • Add analyzer suppression or comment if intentional (unlikely)

Task 3: Audit and Eliminate Async Void Methods

Priority: Medium
Estimated Effort: Medium
Impact: Reliability, testability, exception handling

Description:

Audit the 50 async void methods in the codebase and convert to async Task where possible. Async void methods have unhandled exception semantics that crash the process, making them unsuitable outside event handlers.

Current State:

  • 50 async void methods detected
  • Most appear to be in test code with VSTHRD100 suppressions
  • Some may be intentional (e.g., event handlers, UI callbacks)

Conversion Pattern:

// Before
public async void ProcessAsync()
{
    await DoWorkAsync();
}

// After
public async Task ProcessAsync()
{
    await DoWorkAsync();
}

Tasks:

  1. Identify all 50 async void methods
  2. Categorize: event handlers (keep) vs regular methods (convert)
  3. Convert non-event-handler methods to async Task
  4. For legitimate event handlers, add try-catch with logging
  5. Update callers to await Task-returning methods

Acceptance Criteria:

  • Document all async void methods and justify retention
  • Convert non-event-handler async void to async Task
  • Add exception handling to remaining async void event handlers
  • Update VSTHRD100 suppressions with justification comments

Task 4: Standardize Custom Exception Constructors

Priority: Medium
Estimated Effort: Small
Impact: API consistency, serialization, exception propagation

Description:

Add standard exception constructor overloads to custom exception types following the .NET exception design guidelines. Several custom exceptions (AdapterSettingsException, TestFailedException, etc.) lack the recommended constructors.

Standard Exception Pattern:

public class CustomException : Exception
{
    public CustomException() { }
    
    public CustomException(string message) 
        : base(message) { }
    
    public CustomException(string message, Exception innerException) 
        : base(message, innerException) { }
}

Affected Custom Exceptions:

  • AdapterSettingsException.cs β€” Missing parameterless and innerException constructors
  • TestFailedException.cs β€” Missing constructors
  • InvalidManagedNameException.cs β€” Missing constructors
  • MSTestException.cs β€” Verify constructor completeness

Acceptance Criteria:

  • All custom exceptions have 3 standard constructors
  • Serialization constructors present where needed (check .NET 8 requirements)
  • XML documentation on all custom exception classes
  • Unit tests verify constructor behavior

Task 5: Reduce Hardcoded Exception Messages via Localization

Priority: Medium
Estimated Effort: Large
Impact: Localization, user experience, message consistency

Description:

Reduce the 199 hardcoded exception messages by moving them to resource strings. Currently 62% of exception messages are hardcoded, violating the repository's localization guidelines.

Current Pattern (199 occurrences):

throw new InvalidOperationException("The test context is not available.");

Localized Pattern (121 occurrences):

throw new InvalidOperationException(Resources.TestContextNotAvailable);

Approach:

  1. Audit the 199 hardcoded messages
  2. Identify user-facing vs internal/debug messages
  3. Add resource entries for user-facing messages
  4. Update exception throw sites to use resources
  5. Run dotnet msbuild <project>.csproj /t:UpdateXlf to regenerate .xlf files

Target Areas:

  • TestFramework user-facing exceptions (high priority)
  • Platform services adapter exceptions (medium priority)
  • Internal exceptions can remain hardcoded if never user-visible

Acceptance Criteria:

  • Reduce hardcoded rate from 62% to <40%
  • All public API exceptions use localized resources
  • Update .xlf files with new resource strings
  • Document which exceptions can remain hardcoded (internal-only)

πŸ“Š Historical Context

Previous Focus Areas
Date Focus Area Type
2026-04-30 MSTest Analyzer Ecosystem Health Custom
2026-05-01 Technical Debt Comment Hygiene Custom
2026-05-05 Warning Suppression Debt & Diagnostic Hygiene Custom
2026-05-21 Exception Handling & Error Recovery Patterns Custom

Statistics:

  • Total quality improvement runs: 4
  • Custom focus area rate: 100%
  • Unique areas explored: 4

🎯 Recommendations

Immediate Actions (This Week)

  1. Fix throw ex; Anti-Pattern β€” Priority: High

    • 5 instances in ClassCleanupManager.cs
    • Simple search-replace fix
    • Immediate debugging improvement
  2. Begin ArgumentNullException.ThrowIfNull Migration β€” Priority: High

    • Start with high-traffic public APIs
    • Update coding guidelines document
    • Add to PR review checklist

Short-term Actions (This Month)

  1. Audit Async Void Methods β€” Priority: Medium

    • Categorize all 50 instances
    • Convert non-event-handlers to async Task
    • Document legitimate async void usage
  2. Standardize Custom Exception Constructors β€” Priority: Medium

    • Quick wins on 4-5 custom exception types
    • Improves API consistency
    • Enables better exception propagation
  3. Begin Exception Message Localization β€” Priority: Medium

    • Start with TestFramework public APIs
    • Document localization policy for new code
    • Reduce hardcoded rate incrementally

Generated by Repository Quality Improvement Agent
Next analysis: 2026-05-22 β€” Focus area selected based on diversity algorithm

Generated by Repository Quality Improver Β· ● 3.7M Β· β—·

  • expires on May 23, 2026, 10:57 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/automationCreated or maintained by an agentic workflow.type/tech-debtCode health, refactoring, simplification.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions