Skip to content

perf: engine discovery/init path cleanup (#5528 C)#5611

Merged
thomhurst merged 8 commits intomainfrom
perf/engine-discovery-init
Apr 18, 2026
Merged

perf: engine discovery/init path cleanup (#5528 C)#5611
thomhurst merged 8 commits intomainfrom
perf/engine-discovery-init

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

@thomhurst thomhurst commented Apr 17, 2026

Part C of three-PR split for issue #5528.

Scope

Items 7-10 from the perf checklist — discovery/init path cleanup.

Changes

Item 7 — TestGroupingService sync hot path

GroupTestsByConstraintsAsync previously built an async state machine per call even though every await was behind an IsTraceEnabled guard. The core grouping work is now synchronous; trace messages are collected into a list and flushed through a thin async wrapper only when tracing is enabled. Hot path: zero state-machine overhead, no list allocation for trace messages.

Item 8 — LINQ chains → direct loops

  • TestSessionCoordinator.InitializeEventReceivers — dropped Select(t => t.Context) and passed the test list through directly.
  • EventReceiverOrchestrator.InitializeTestCounts — accepts IReadOnlyList<AbstractExecutableTest> directly; no more allTestContexts as IList<TestContext> ?? ToList() fallback. Internal-only API, no public surface change.
  • EventReceiverOrchestrator.InvokeHookRegistrationEventReceiversAsync — replaced OfType<T>().OrderBy(...) with an explicit filtering loop + in-place List<T>.Sort.
  • HookDelegateBuilder — three copies of .OrderBy().ThenBy().Select().ToList() consolidated into a single SortAndProject<TContext> helper using in-place List.Sort + index loop.
  • TestDiscoveryContext.Assemblies / .TestClassesSelect(...).Distinct().ToArray() replaced with inlined HashSet<T>.Add loops + CopyTo.
  • TestDiscoveryService.GetCachedTestContexts — materializes a List<TestContext> from the cached queue instead of returning a Select enumerable that allocates a lambda-backed iterator per foreach.

Item 9 — TestSessionContext.HasFailures via atomic counter

Added public bool HasFailures backed by a lock-free int _failureCount incremented via Interlocked.Increment in TestCoordinator's terminal switch for Failed/Timeout/Cancelled states. HookExecutor.ExecuteAfterTestSessionHooksAsync now reads the flag in O(1) instead of materializing AllTests (which itself eagerly flattens SelectMany across every class) and LINQ-scanning it.

Item 10 — TestContext.Id lazy string

TestContext now stores a Guid field and materializes the string form lazily via field ??= _idGuid.ToString(). The internal registry is keyed by Guid (ConcurrentDictionary<Guid, TestContext>); GetById(string) parses once on the lookup path. TestCoordinator cleanup calls the new RemoveFromRegistry() instance method so a test's string Id is never materialized unless something external asks for it. Public surface unchanged.

Verification

  • Build clean across all TFMs.
  • TUnit.Engine.Tests (net10.0): 148/148 passed, 99 skipped (platform/CI-only).
  • TUnit.UnitTests (net10.0): 180/180 passed.
  • TUnit.PublicAPI (net10.0): 4/4 passed. One expected snapshot update (HasFailures property added to TestSessionContext) committed in Tests.Core_Library_Has_No_API_Changes.DotNet10_0.verified.txt.

Related PRs

Scaffolding commit for PR C. Will contain:

- #7 TestGroupingService: make sync, move trace logging behind guard

- #8 LINQ chains in discovery/init replaced with direct loops

- #9 TestSessionContext.HasFailures via atomic counter (O(1))

- #10 TestContext.Id lazy string from stored Guid or counter
@codacy-production
Copy link
Copy Markdown

codacy-production bot commented Apr 17, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

TIP This summary will be updated as you push new changes. Give us feedback

@claude
Copy link
Copy Markdown
Contributor

claude bot commented Apr 17, 2026

Code Review — PR #5611 (Part C, Draft Placeholder)

Status: No code to review yet — this PR contains only a scaffolding commit with 0 file changes. The review below covers the planned changes described in the PR description.


Planned Item #7TestGroupingService.GroupTestsByConstraintsAsync: drop async state machine

Good call. If the sort is truly synchronous, wrapping it in an async state machine adds unnecessary overhead (allocation of IAsyncStateMachine, AsyncTaskMethodBuilder, etc.). Returning ValueTask.CompletedTask or making the method return void/sync will be a clean win. Just verify no callers await it in a way that depends on yielding back to the caller mid-operation.

Planned Item #8 — Replace LINQ chains with direct loops/sorts

Solid micro-optimization for hot discovery paths. One architectural note: when replacing LINQ, prefer List<T>.Sort() with a Comparison<T> delegate over OrderBy().ToList() to avoid intermediate allocations. If the collection size is small and bounded, also consider whether sorting is even needed vs. insertion-order maintenance.

Planned Item #9TestSessionContext.HasFailures: atomic counter (O(1))

This is the highest-value change in this set. O(N) traversal on every HasFailures check is a correctness-risk under concurrent test runs, not just a perf issue. Using Interlocked.Increment in TestStateManager.MarkFailed is exactly right. A few things to confirm when the code lands:

  • The counter must be incremented before the state is observable to other threads (memory ordering).
  • Consider whether a decrement path is needed if a test transitions out of failed state (e.g., retry mechanisms).
  • Expose as bool HasFailures => _failureCount > 0 (volatile read or Interlocked.Read) rather than the counter value directly, to keep the API surface clean.

Planned Item #10TestContext.Id: store Guid, lazy .ToString()

Good. Guid.ToString() is non-trivial (allocates a 36-char string). Storing the Guid and deferring .ToString() until actually needed is idiomatic for perf-sensitive paths. If the Id is only ever used for logging/display (not as a dictionary key on the string form), you might also consider a monotonic long counter via Interlocked.Increment — cheaper to allocate, compare, and format than a Guid.


Overall

The planned scope is well-targeted: each item has a clear, measurable benefit and a narrow blast radius. No architectural concerns with the approach. Looking forward to reviewing the actual implementation once Parts A and B are approved.

Nothing to block on yet — come back when code lands.

- TestGroupingService: hot path now fully synchronous; trace-message
  emission moved behind an IsTraceEnabled guard. Eliminates the async
  state machine when tracing is off (the normal case).
- LINQ chains replaced with direct loops in EventReceiverOrchestrator,
  HookDelegateBuilder, TestDiscoveryContext, TestDiscoveryService, and
  TestSessionCoordinator. Also switched InitializeTestCounts to accept
  IReadOnlyList<AbstractExecutableTest> directly, dropping the Select
  projection + ToList fallback.
- TestSessionContext: added HasFailures backed by a lock-free failure
  counter incremented from TestCoordinator on Failed/Timeout/Cancelled.
  Replaces an O(N) AllTests.Any() traversal in the after-session hook
  path.
- TestContext.Id: Guid is stored and the string form is materialized
  lazily. The registry is keyed by Guid internally; GetById(string)
  parses once. Removes the eager ToString allocation for tests that
  never access the string Id.
@thomhurst thomhurst marked this pull request as ready for review April 17, 2026 22:41
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #5611 (Part C: Discovery/Init Path Cleanup)

Following up on the prior placeholder comment from when this PR had no code. All four items are now implemented — reviewing the actual changes.


Overall

Well-targeted perf work with a narrow blast radius. Each change is individually motivated and the implementation is clean. A few observations below, none blocking.


Item 7 — TestGroupingService: hot/cold path split

Good. Returning new ValueTask<GroupedTests>(result) on the hot path avoids the async state-machine allocation entirely. The tracing path is correctly isolated to GroupTestsByConstraintsWithTracingAsync.

One observable behavior change worth noting: trace messages are now buffered in a List<string> and flushed after grouping completes, rather than emitted inline as each test is categorized. Under concurrent test runs, this means grouping traces now appear as a batch, potentially out-of-order relative to other concurrent log output. This is an acceptable trade-off for the perf win, but could make trace-based debugging slightly harder — worth a note in the change description if you haven't already.


Item 8 — LINQ → loops + SortAndProject

Good. SortAndProject<TContext> is clean — HookDelegateBuilder is non-generic, so the method's own type parameter is unambiguous with no shadowing concern.

Observed: the existing BuildBefore/AfterTest/ClassHooksAsync methods already use a SortAndAddHooks helper (not touched by this PR), while the three assembly/global methods used raw LINQ. This PR completes the consistency.

Double-sort in InvokeHookRegistrationEventReceiversAsync is intentional and preserved correctly — the comment explains why (FilterScopedAttributes keyed on first-encountered order), no issue here.


Item 9 — HasFailures atomic counter

Excellent. Volatile.Read(ref _failureCount) > 0 is correctly implemented, and Interlocked.Increment in MarkFailure() provides the necessary memory-order guarantees. Replacing the O(N) AllTests LINQ traversal in HookExecutor with this O(1) read is the highest-value change in this PR.

One edge case to confirm: Cancelled tests also call MarkFailure() — this matches the original Any(t => t.Result is { State: Failed or Timeout or Cancelled }) check in HookExecutor, so semantics are preserved.


Item 10 — TestContext.Id lazy string

Mostly good. The Guid-as-primary-key approach is correct and the registry change is clean. However:

Potential concern with _idString ??= _idGuid.ToString(): This compound null-check-and-assign is not atomic. Under concurrent access, two threads can both observe _idString == null, both call _idGuid.ToString(), and each write its own allocation. The result is correct (both strings are identical) and reference assignment is atomic on x64, but _idString is not volatile, so a thread may read a stale null. On strong-memory-order platforms (x64) this is benign in practice; on platforms with weaker guarantees (ARM, WASM), or under AOT compilation without JIT memory-fence insertion, it is technically not safe.

Given TUnit's AOT-compatibility requirement, consider one of:

  • Marking _idString as volatile (simplest fix, no API change)
  • Using Interlocked.CompareExchange(ref _idString, _idGuid.ToString(), null) if strict once-init matters
  • Or explicitly document the intentional benign-race pattern if the current behavior is acceptable

Note: The retained static RemoveById(string id) now does a Guid.TryParse on every call. If external callers use it, they pay a parse cost that didn't exist before. If it's internal-only with no remaining callers (the coordinator now uses RemoveFromRegistry() directly), consider removing RemoveById to avoid confusion. If it must stay for API surface reasons, the Guid.TryParse overhead is fine for a non-hot path.


Summary

Item Status
Item 7 — Sync hot path in TestGroupingService ✅ Good, minor trace-ordering note
Item 8 — LINQ → SortAndProject ✅ Clean and complete
Item 9 — HasFailures atomic counter ✅ Excellent
Item 10 — Lazy Id string + Guid registry ✅ Good, volatile concern on non-x64/AOT

No blocking issues. The volatile concern on _idString is the only thing I'd fix before merge, especially given the AOT compatibility mandate.

Addresses review feedback on #5611: `_idString ??= _idGuid.ToString()`
is not safe under weak memory ordering (ARM/WASM/AOT). A reader could
observe a stale null even after another thread wrote the string. Mark
the field volatile — reference writes are atomic, so a benign double
ToString is possible but harmless; volatile fixes the staleness.
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up Review — PR #5611

Correction from prior review

The previous review stated _idString is not volatile and suggested adding it. That was a misread — the diff already declares:

private volatile string? _idString;

The implementation is correct: volatile ensures ARM/WASM/AOT visibility, the benign double-ToString race is documented inline, and the comment explains why it's acceptable. No action needed on Item 10.


New observation: HashSet ordering in BuildUniqueAssemblies / BuildUniqueTestClasses

The replaced code used Select().Distinct().ToArray() — LINQ's Distinct() is documented to preserve first-occurrence insertion order. The new code uses HashSet<T> + CopyTo, which is not contractually ordered by the HashSet<T> documentation (the spec says "unordered").

In practice, the current .NET HashSet<T> implementation does iterate in insertion order, so the behavior is identical today. However, this is an implementation detail that could change. If the order of Assemblies or TestClasses affects hook execution sequencing, this is worth a brief comment or a switch to LinkedHashSet-equivalent (e.g., an OrderedDictionary-backed dedup, or just a List + HashSet pair). Low risk today, but flagging it.


Minor: null-check inconsistency in GroupTestsByConstraintsCore

Three of the four tracing branches use the null-conditional form:

traceMessages?.Add(...);

But the notInParallel != null branch uses an explicit block:

if (traceMessages != null) { ...; traceMessages.Add(...); }

The explicit block is needed there because it also computes keys (which is only needed for the message). This is correct as-is — just noting it's intentional, not an inconsistency.


Overall

The prior review's volatile concern is a non-issue — it was already addressed in the code. The HasFailures atomic counter (Item 9) and SortAndProject consolidation (Item 8) are the strongest changes here. The HashSet ordering note is low risk but worth keeping in mind for long-term maintainability. No blocking issues.

CI failed on net8.0/net9.0/net4_7 Core PublicAPI diff at line 1684 —
the HasFailures property added in the prior commit was only reflected
in the net10.0 snapshot. Add the same entry to the other framework
variants so the PublicAPI verification passes across all TFMs.
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: perf engine discovery/init path cleanup (Part C)

Solid, targeted performance work across the discovery/init path. Most changes are straightforward and well-executed. One semantic concern is worth investigating before merging.


Concern: HasFailures does not account for retries

File: TUnit.Core/Models/TestSessionContext.cs, TUnit.Engine/Services/TestExecution/TestCoordinator.cs

MarkFailure() is called eagerly when a test transitions to Failed/Timeout/Cancelled. There is no corresponding decrement if the test is later retried and passes.

The original O(N) scan read final result state at the time the after-session hook ran:

// Original — reflects final outcome after retries complete
AllTests.Any(t => t.Result is { State: TestState.Failed or Timeout or Cancelled })

With the new implementation, any test that fails an intermediate retry attempt permanently sets HasFailures = true, even if the session ultimately ends with zero failed tests. Users relying on after-session hooks that branch on HasFailures (e.g. failure notifications, conditional teardown) would get incorrect behavior when [Retry] is in play.

Suggested approach: Gate MarkFailure() on whether the test is in its truly final state. If TUnit distinguishes "retrying" from "finally failed" in TestCoordinator, call MarkFailure() only in the final-failure branch, not the retry-attempt branch. Alternatively, track a final-pass counter alongside the failure counter and expose HasFailures => _failureCount > _retriedToPassCount.

This needs verification — if retried tests always produce a fresh TestContext so the original Failed context is never in AllTests by the time the hook runs, the concern may not apply and the semantics are equivalent. Worth a quick confirmation either way.


Minor: volatile + ??= on TestContext.Id — the comment is doing heavy lifting

File: TUnit.Core/TestContext.cs:52

The multi-line comment explaining why volatile is needed here is good, but the design itself is fragile. Two threads can both observe null, both materialize the string, and both write — which is benign only because Guid.ToString() on the same Guid always yields equal strings. If a future maintainer changes the ID format to include a counter or timestamp, this becomes a silent correctness bug.

Lazy<string> or Interlocked.CompareExchange(ref _idString, _idGuid.ToString(), null) makes the intent explicit and eliminates the race documentation burden:

public string Id => Interlocked.CompareExchange(ref _idString, _idGuid.ToString(), null) 
                    ?? _idString!;

Or more idiomatically with LazyInitializer.EnsureInitialized. The performance difference is negligible for a field that is accessed at most once per test in normal flow.


What's well done

  • TestGroupingService split — the ValueTask<GroupedTests> fast-path using return new ValueTask<GroupedTests>(...) for the synchronous case is the right pattern. Zero state-machine overhead on the hot path, and the null-conditional traceMessages?.Add(...) correctly short-circuits string interpolation evaluation when tracing is disabled.

  • SortAndProject<TContext> helper — three identical sort+project chains reduced to one. The List.Sort with a Comparison<T> delegate matches the original OrderBy.ThenBy semantics exactly (stable sort, same key precedence), so this is a safe consolidation.

  • Volatile.Read(ref _failureCount) on the reader side — correct. Interlocked.Increment provides a full fence on the write side; Volatile.Read is the minimal correct fence on the read side. No over-synchronization.

  • RemoveFromRegistry() instance method — avoiding string materialization in the cleanup path is a nice touch; the string Id property was the only path that allocated a string from the Guid, and cleanup doesn't need it.

  • InitializeTestCounts accepting IReadOnlyList<AbstractExecutableTest> — eliminates the intermediate Select projection and avoids the hidden ToList() that the old as IList<T> fallback forced. The for (var i = 0; ...) loop is preferable here since the index avoids an enumerator allocation.

Only consumer is HookExecutor (engine-internal) — no user-facing need to
expose the failure-count gate. Dropping to `internal` shrinks the public
surface, removes the PublicAPI snapshot entries across all four TFMs,
and keeps the counter representation free to evolve.
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #5611: Engine Discovery/Init Path Cleanup (Part C)

This is a well-scoped, performance-focused PR. The changes are individually motivated and most implementations are correct. I have one semantic concern about retry correctness, a note on hash-set ordering, and a few minor observations below.


TestGroupingService: Sync/async hot-path split

Verdict: Good, with one behavioral note.

The split is correctly implemented. Returning new ValueTask<GroupedTests>(result) for the non-tracing hot path eliminates the async state-machine allocation entirely — this is the right pattern for ValueTask optimization.

Behavioral change worth noting: trace messages are now buffered in a List<string> and emitted after all grouping completes, rather than interleaved inline. Under parallel test-session output, grouping traces will appear as a batch at the end of the grouping phase. Fine for production use, but worth noting if anyone uses trace logs for debugging ordering issues.

Minor inconsistency: Three branches use traceMessages?.Add(...) (null-conditional), while the notInParallel != null branch uses an explicit if (traceMessages != null) block. The explicit block is correct there (it avoids allocating keys when tracing is off), but a short comment would make the asymmetry look intentional rather than accidental.


LINQ → direct loops + SortAndProject

Files: HookDelegateBuilder.cs, EventReceiverOrchestrator.cs, TestDiscoveryService.cs, TestSessionCoordinator.cs

Verdict: Clean and correct.

SortAndProject<TContext> consolidates three identical sort+project chains. List<T>.Sort with a Comparison<T> delegate is a stable sort with the same semantics as OrderBy(...).ThenBy(...), so this is a safe refactor.

TestDiscoveryContext.BuildUniqueAssemblies/BuildUniqueTestClasses — ordering caveat:

// Before: LINQ Distinct() — documents first-occurrence insertion order
TestClasses.Select(x => x.AssemblyContext).Distinct().ToArray()

// After: HashSet<T> + CopyTo — iteration order is an implementation detail, not guaranteed
var seen = new HashSet<AssemblyHookContext>();
...
seen.CopyTo(result);

In practice, .NET's HashSet<T> iterates in insertion order for small sets, but the framework specification does not guarantee it. If the order of Assemblies or TestClasses affects hook execution sequencing (e.g., assembly-level before-hooks run in assembly order), a later .NET runtime change could silently reorder things. Low risk today, but if order matters here, consider a List<T> + HashSet<T> pair (append to list only when seen.Add returns true) to get both deduplication and guaranteed insertion-order iteration.


TestSessionContext.HasFailures atomic counter

Files: TestSessionContext.cs, TestCoordinator.cs, HookExecutor.cs

Verdict: Semantically correct — but one edge case worth verifying.

Volatile.Read + Interlocked.Increment is the right primitive pair here. The O(1) replacement of the O(N) AllTests.Any(...) scan in ExecuteAfterTestSessionHooksAsync is the highest-value change in this PR.

Retry semantics: MarkFailure() is reached only after all retry attempts are exhausted (since RetryHelper.ExecuteWithRetry absorbs intermediate failures and only re-throws on the last attempt, which propagates to catch → MarkFailed → switch). Semantics are equivalent to the original scan.

One edge case not covered: MarkCircularDependencyFailed and MarkDependencyResolutionFailed in TestStateManager set test.State = TestState.Failed but are invoked from TestDependencyResolver, which does not route through the finally-block switch. These tests will appear as Failed in the results without ever calling MarkFailure(). If HasFailures is used to decide whether to flag the session as having errors, dependency-resolution failures would not increment the counter. Worth explicitly verifying this is the intended behavior.


TestContext.Id lazy string + Guid registry

File: TUnit.Core/TestContext.cs

Verdict: Correct.

volatile string? _idString + _idString ??= _idGuid.ToString() is correct: volatile ensures cross-thread visibility on weak-memory-order platforms (ARM, WASM, AOT), the benign double-ToString race is harmless because two Guid.ToString calls on the same Guid produce equal strings, and reference assignment is atomic. The inline comment documenting this reasoning is exactly right.

RemoveById(string id) — consider removing if unused:

internal static void RemoveById(string id)
{
    if (Guid.TryParse(id, out var guid))
        _testContextsByGuid.TryRemove(guid, out _);
}

If there are no remaining callers (the coordinator now calls RemoveFromRegistry() directly), this method should be removed to avoid confusion. If it must stay, note that a caller passing an invalid (non-Guid) string will now silently do nothing — make sure that's the intended behavior.


Summary

Area Verdict Action
Sync hot path + buffered trace messages ✅ Good Minor: comment on if vs ?. asymmetry
SortAndProject + loop refactors ✅ Clean Consider List+HashSet pair if assembly/class order matters for hooks
HasFailures atomic counter ✅ Correct Verify dependency-resolution failures path also triggers MarkFailure
Lazy Id string + Guid registry ✅ Correct Consider removing unused RemoveById

No blocking issues. The most substantive thing to verify before merge is whether MarkCircularDependencyFailed/MarkDependencyResolutionFailed paths need to call MarkFailure() as well.

@thomhurst
Copy link
Copy Markdown
Owner Author

Thanks for the follow-up review. Addressing both points:

Re: HasFailures + retries — not an issue. RetryHelper.ExecuteWithRetry catches intermediate exceptions inside its own loop and only rethrows on final failure. The MarkFailure() call in TestCoordinator lives in the post-lifecycle switch (test.State) block, which runs once per test after all retry attempts complete. So test.State reflects the final outcome and MarkFailure() is never called for intermediate retry failures. Semantics match the original AllTests.Any(...) scan exactly.

Re: TestContext.Id lazy materialization — fair point. Switched to Interlocked.CompareExchange so we publish a single string even if the Id format ever becomes non-deterministic. Removes the documentation burden too.

Replaces `volatile` field + `??=` with `Volatile.Read` + `CompareExchange`
so only one string is ever published even if the Id format becomes
non-deterministic in the future. ARM/WASM acquire semantics preserved
via `Volatile.Read` on the fast path.
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #5611 (follow-up on latest commit 6f36c34)

Previous review concerns — status update

TestContext.Id lazy string (was: volatile concern): The latest commit (6f36c34) correctly replaces volatile string? _idString with a manual Volatile.Read + Interlocked.CompareExchange pattern:

// Fast path: acquire-fence read avoids state machine on ARM/WASM/AOT
if (Volatile.Read(ref _idString) is { } existing)
    return existing;

var materialized = _idGuid.ToString();
return Interlocked.CompareExchange(ref _idString, materialized, null) ?? materialized;

This is the correct lock-free lazy-init pattern — same as LazyInitializer.EnsureInitialized<T> internally. Volatile.Read provides the acquire fence; Interlocked.CompareExchange provides the full fence on write. The return expression is correct in both races (CAS winner returns materialized, loser returns the existing string from the old value). This fully resolves the previous concern and the suggestion.


Open items from prior reviews — still unresolved

MarkFailure() not reached via dependency-resolution failure paths:

MarkCircularDependencyFailed and MarkDependencyResolutionFailed in TestStateManager set test.State = TestState.Failed but are invoked from TestDependencyResolver, which bypasses the finally-block switch in TestCoordinator. These tests will appear as Failed in the final result set without ever calling MarkFailure().

Original behavior: AllTests.Any(t => t.Result is { State: Failed or Timeout or Cancelled }) would catch these because it reads final Result.State at after-session hook time.
New behavior: HasFailures returns false for sessions where only dependency-resolution failures occurred.

This is the one behavioral change worth explicitly confirming or fixing before merge.

HashSet ordering in BuildUniqueAssemblies/TestClasses:

HashSet<T>.CopyTo ordering is an implementation detail, not a contract. LINQ's Distinct() preserves first-occurrence order by spec. If assembly/class hook execution order is driven by the order of Assemblies/TestClasses, a future runtime could silently reorder. Low risk today — noting it's still unaddressed.


Summary

Item Status
Lazy Id + Volatile.Read + CAS pattern ✅ Resolved — clean implementation
HasFailures retry semantics ✅ Verified correct (retries absorbed before switch)
HasFailures dependency-resolution paths ⚠️ Not reached via MarkCircularDependencyFailed — needs verification
HashSet ordering for Assemblies/TestClasses ⚠️ Acknowledged but unaddressed
RemoveById unused after RemoveFromRegistry Minor — cleanup candidate

The Interlocked.CompareExchange change is the right call. The dependency-resolution failure path is the only open correctness question.

- TestScheduler: call MarkFailure() on circular-dep failures so
  HasFailures reflects them (prior O(N) AllTests scan caught these;
  counter-based HasFailures missed them since circular-dep tests
  skip the TestCoordinator lifecycle switch)
- TestDiscoveryContext: BuildUniqueAssemblies/TestClasses use
  List+HashSet pair to guarantee first-occurrence insertion order
  (matches prior LINQ Distinct() contract; HashSet.CopyTo ordering
  is an implementation detail)
- TestContext: remove unused internal RemoveById; callers now use
  RemoveFromRegistry() directly
Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #5611 (follow-up on commits 7a6a7b8 + da0a357)

All previous open items resolved

** path (was: ⚠️ open) — ✅ Fixed**
TestScheduler.cs now calls TestSessionContext.Current?.MarkFailure() immediately after MarkCircularDependencyFailed. Circular-dep failures correctly increment the counter.

MarkDependencyResolutionFailed path (was: mentioned as open) — ✅ Confirmed non-issue
These tests proceed through TestCoordinator.ExecuteTestInternalAsync. Because test.Result is directly backed by Context.Execution.Result (AbstractExecutableTest.cs:48), the pre-set Failed result is caught by the existingResult?.State == TestState.Failed guard at L82, which calls MarkFailed (setting test.State = TestState.Failed). The finally-block switch then reaches case TestState.Failed:MarkFailure(). No gap here.

HashSet ordering in BuildUniqueAssemblies/TestClasses (was: ⚠️ open) — ✅ Fixed
Now uses a List<T> + HashSet<T> pair: seen.Add returns false on duplicates, ordered.Add keeps only first-occurrence entries. Insertion order is now contractually guaranteed, matching the prior LINQ Distinct() semantics.

Unused RemoveById (was: minor) — ✅ Removed

BuildUniqueAssemblies comment (latest commit da0a357) — ✅ Clarified
Comment now explains why the List is needed (HashSet iteration order is an implementation detail, not a contract).


Summary

All concerns from prior reviews have been addressed. The implementation is correct, the edge cases are handled, and the perf wins (atomic HasFailures counter, lazy Id string, SortAndProject consolidation, sync hot path in grouping service) are sound. No blocking issues. ✅

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #5611 (follow-up on commits 7a6a7b8 + da0a357)

All previous open items resolved

MarkCircularDependencyFailed path (was: open) — Fixed
TestScheduler.cs now calls TestSessionContext.Current?.MarkFailure() immediately after MarkCircularDependencyFailed. Circular-dep failures correctly increment the counter.

MarkDependencyResolutionFailed path (was: mentioned as open) — Confirmed non-issue
These tests proceed through TestCoordinator.ExecuteTestInternalAsync. Because test.Result is directly backed by Context.Execution.Result (AbstractExecutableTest.cs:48), the pre-set Failed result is caught by the existingResult?.State == TestState.Failed guard at L82, which calls MarkFailed (setting test.State = TestState.Failed). The finally-block switch then reaches case TestState.Failed: => MarkFailure(). No gap here.

HashSet ordering in BuildUniqueAssemblies/TestClasses (was: open) — Fixed
Now uses a List<T> + HashSet<T> pair: seen.Add returns false on duplicates, ordered.Add keeps only first-occurrence entries. Insertion order is now contractually guaranteed, matching the prior LINQ Distinct() semantics.

Unused RemoveById (was: minor) — Removed

BuildUniqueAssemblies comment (latest commit da0a357) — Clarified
Comment now explains why the List is needed (HashSet iteration order is an implementation detail, not a contract).


Summary

All concerns from prior reviews have been addressed. The implementation is correct, the edge cases are handled, and the perf wins (atomic HasFailures counter, lazy Id string, SortAndProject consolidation, sync hot path in grouping service) are sound. No blocking issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant