Skip to content

Fix flaky CounterSampleCalculator_ElapsedTime test#127204

Merged
danmoseley merged 4 commits intodotnet:mainfrom
danmoseley:fix-elapsedtime-flaky-test
Apr 21, 2026
Merged

Fix flaky CounterSampleCalculator_ElapsedTime test#127204
danmoseley merged 4 commits intodotnet:mainfrom
danmoseley:fix-elapsedtime-flaky-test

Conversation

@danmoseley
Copy link
Copy Markdown
Member

@danmoseley danmoseley commented Apr 21, 2026

I got this on an unrelated PR.

Note

This PR was authored with assistance from Copilot CLI.

The CounterSampleCalculator_ElapsedTime test occasionally fails in CI (0.01% rate) because it compares elapsed time measured by two different clock sources: QPC (via the perf counter infrastructure) and DateTime.Now (over system clock, updated by periodic interrupt). Under CI load, the gap between Stopwatch.GetTimestamp() and DateTime.Now on consecutive lines can exceed the 0.3s tolerance.

Assert.True() Failure
Expected: True
Actual:   False
   at System.Diagnostics.Tests.CounterSampleCalculatorTests.CounterSampleCalculator_ElapsedTime() in /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/CounterSampleCalculatorTests.cs:line 29

Fix by using Stopwatch for both measurements so they share the same clock source (QPC). Can't use .GetElapsedTime() as that wasn't in .NET Framework.

Use Stopwatch for both elapsed time measurements so they share the
same clock source (QPC), instead of comparing QPC-based counter values
against DateTime.Now. Under CI load the gap between the two clock
sources could exceed the 0.3s tolerance.

Also removed unused using directives.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @dotnet/area-system-diagnostics-tracing
See info in area-owners.md if you want to be subscribed.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Addresses flakiness in CounterSampleCalculator_ElapsedTime by ensuring elapsed-time comparisons use a consistent clock source.

Changes:

  • Replaced DateTime.Now-based elapsed measurement with a Stopwatch-based calculation.
  • Removed unused using directives.
Show a summary per file
File Description
src/libraries/System.Diagnostics.PerformanceCounter/tests/CounterSampleCalculatorTests.cs Updates the elapsed-time test to compute expected duration using Stopwatch (QPC) and cleans up unused usings.

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for fixing another flaky test @danmoseley !

Copy link
Copy Markdown
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the gap between Stopwatch.GetTimestamp() and DateTime.Now on consecutive lines can exceed the 0.3s tolerance.

This does not make sense. Both of these have sub-millisecond granularity.

I agree that tests with small hardcoded time spans are prone to failing intermittently on overloaded machines due to thread scheduling happening at the right moment. Changing the time source won't fix these failures. The fix for these failures should be to either increase the hardcoded time span or to add a retry loop.

@danmoseley
Copy link
Copy Markdown
Member Author

I had read that the system clock was updated on a 15ms interrupt, while the fine grained timer was not?

I'll add a loop.

Wrap the timing-sensitive measurement and assertion in
RetryHelper.Execute (3 attempts) so transient CI scheduling
delays don't cause flaky failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 21, 2026 15:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 2

@jkotas
Copy link
Copy Markdown
Member

jkotas commented Apr 21, 2026

I had read that the system clock was updated on a 15ms interrupt,

It might in some cases, but that's still 20x less than 0.3 seconds that we allow currently.

Wrap test body in try/finally so DeleteCategory and Dispose
always run even if retries are exhausted. Restrict retries to
XunitException (assertion failures) so unexpected errors
propagate immediately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
Copy link
Copy Markdown
Member Author

Yeah I looped against the Windows API's on a machine that was artificially loaded (quite different to CI hardware no doubt)

GetSystemTimePreciseAsFileTime, QueryPerformanceCounter, DateTime.UtcNow.Ticks diverged by up to 0.1-0.2ms only.

So I guess the cause was scheduling causing the perf counter read and the our time read to be separated by 30ms.

@danmoseley danmoseley enabled auto-merge (squash) April 21, 2026 16:21
@danmoseley danmoseley merged commit 335fa34 into dotnet:main Apr 21, 2026
91 of 96 checks passed
@danmoseley danmoseley deleted the fix-elapsedtime-flaky-test branch April 21, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants