feat(justdummies): add JD018-JD022 for the seeding long tail - #373
Merged
Conversation
Closes the reproducibility family with the five remaining ways a seed fails to mean what the code says it means. JD018 reports a reproducibility scope nested inside another. The premise was verified before the rule was written: Any.Reproducibly takes its seed from Guid.NewGuid().GetHashCode(), not from the ambient source, so an inner scope ignores whatever the outer one pinned. The outer mechanism still reports its own seed, so the failure names a seed that reproduces nothing — a wrong instruction rather than a wrong result. The seeded overload is left alone: pinning a chosen seed inside is deliberate. JD019 reports a constant replay seed left in committed code, and ships opt-in on measurement rather than taste. Enabled across JustDummies.UnitTests it reports 238 sites, nearly all legitimate: this repository's own maintainer guide instructs "pin a seed for anything statistical", so a default-on rule would fight documented practice. It earns its keep as a pre-release sweep somebody turns on deliberately. JD020 reports an AnyContext held in a static field — the shape that looks maximally deterministic and is not, since interleaved draws make neither the sequence nor the multiset stable. Info, because a single-threaded consumer shares one harmlessly and the rule cannot see the suite's parallelism configuration. JD021 reports a blank replay snippet. The guard rejects it at run time, but from an adapter hook that runs before every test — so the throw fails the whole suite as an infrastructure error rather than one assertion. JD022 reports a parallel work item that draws without opening its own seed scope. It carries no code fix by design: the repair needs a run seed the developer must choose and record, and inventing one would produce exactly the committed-seed problem JD019 describes. All five were confirmed by a planted positive control in a real build, which also established that JD020 and JD022 are invisible at default verbosity because they are Info — worth knowing before reading a clean sweep as evidence of anything. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the
JustDummies.Reproducibilityfamily with the five remaining ways a seed fails to mean what the code says it means. JustDummies now ships 22 rules.Type of change
Changes
NestedReproducibilityScope(Warning) — a scope nested inside another. Verified before writing the rule:Any.Reproduciblytakes its seed fromGuid.NewGuid().GetHashCode(), not from the ambient source, so an inner scope ignores whatever the outer one pinned and draws afresh every run. The outer mechanism still reports its seed, so the failure names a seed that reproduces nothing — a wrong instruction rather than a wrong result. The seeded overload is left alone: pinning a chosen seed inside is deliberate.CommittedReplaySeed(Info, opt-in) — a constant replay seed left in committed code.SharedStaticAnyContext(Info) — anAnyContextin a static field: looks maximally deterministic, and is not, since interleaved draws make neither the sequence nor the multiset stable. Info because a single-threaded consumer shares one harmlessly and the rule cannot see the suite's parallelism configuration.BlankReplaySnippet(Warning) — the guard rejects it at run time, but from an adapter hook that runs before every test, so the throw fails the whole suite as an infrastructure error rather than one assertion.ParallelDrawWithoutPerItemSeed(Info) — a parallel work item drawing without its own scope. The ambient scope flows into every worker, so one shared scope replays nothing.JD019's opt-in is a measurement, not a preference
Enabled across
JustDummies.UnitTestsit reports 238 sites, nearly all legitimate. This repository's own maintainer guide instructs the opposite for a whole class of tests — "Pin a seed for anything statistical" — so a default-on rule would fight documented practice. It ships as a pre-release sweep somebody turns on deliberately, and the help page carries the enabling snippet.That is the second rule in this series whose default was settled by measuring rather than by reasoning, after JD011.
JD022 carries no code fix, by design
The repair needs a run seed the developer must choose and record, plus a per-item derivation from the loop variable. An analyzer cannot invent a seed someone intends to keep — generating one would produce exactly the committed-seed problem JD019 describes. The help page shows the manual shape instead.
Verification
Zero hits across
JustDummies.UnitTests,JustDummies.PropertyTests,JustDummies.Xunit.UnitTests,FirstClassErrors.Testing.UnitTestsandFirstClassErrors.UnitTests.A planted positive control confirmed all five fire at the expected locations — and established something worth knowing for future waves: JD020 and JD022 are invisible at default build verbosity because they are
Info. A clean sweep over Info rules is not evidence of anything until they are escalated, which is how the control was run.Testing
dotnet build FirstClassErrors.sln— 0 warnings, 0 errorsdotnet test FirstClassErrors.sln— 2 092 tests, 0 failures across all 13 test projectsFirstClassErrors.Analyzers.UnitTests, 132) — the rules added here live inJustDummies.Analyzers.UnitTests(175)Mutation testing was not run locally.
Documentation
doc/updated —JD018–JD022pages and the rule indexJD018.fr.md–JD022.fr.mdand the French indexArchitecture decisions
Proposed: ADR-____These apply ADR-0038 (the ambient seed scope opened to adapters) and ADR-0042 (reproducibility scoped to the draw sequence) rather than deciding anything new.
Remaining
Eight catalogued rules: the scalar-chain emptiness test, no-effect constraints, duplicate pool values, the empty relative URI, the unused
Combineoperand,Distinctover reference equality, plus the two waiting on you — the unsupported-regex rule (needs the grammar shared with the library) and the production-code rule (needs theIsTestProjectMSBuild oracle, withFirstClassErrors.Testingas a live counterexample).🤖 Generated with Claude Code
https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
Generated by Claude Code