test: make EWMA failure-rate cases deterministic with a seeded RNG#26
Merged
Conversation
Two subtests in TestBreaker_Observe_State were named "(EWMA flaky)" and failed intermittently (~13% of runs). The flakiness had two causes: the unseeded global math/rand source, and t.Parallel() subtests all drawing from that shared global source, so even a fixed global seed would interleave nondeterministically. Give each subtest its own *rand.Rand seeded with a fixed testSeed and thread it through failureFunc. The statistical scenarios now exercise the same realistic mixed traffic on every run, deterministically, so the "flaky" labels are dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes flakiness from two EWMA-related subtests in TestBreaker_Observe_State by eliminating shared use of the global math/rand source under t.Parallel(), making the statistically-driven stages deterministic and repeatable.
Changes:
- Introduces a fixed
testSeedconstant for deterministic test randomness. - Refactors stage
failureFunchelpers/signatures to accept a per-subtest*rand.Rand. - Creates a per-subtest RNG (
rand.New(rand.NewSource(testSeed))) so parallel subtests don’t interleave draws.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
totallyunknown
approved these changes
Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the two
TestBreaker_Observe_State/...subtests that were labelled(EWMA flaky)deterministic.Why
They failed intermittently (~13% of runs). Two causes:
math/randsource.t.Parallel()subtests all drawing from that shared global source — so even a fixed global seed would interleave nondeterministically.A flaky test in CI trains everyone to reflexively re-run, which is how a real failure eventually slips through.
How
*rand.Randseeded with a fixedtestSeed, threaded throughfailureFunc.Notes
Test-only change; no production code touched.
🤖 Generated with Claude Code