Skip to content

JustDummies: Combine over two different seeded sources over-promises a full replay #319

Description

@Reefact

Problem

When a composer fails inside a Combine, the AnyGenerationException appends a replay hint. Two levels exist (RandomSource.cs): a full promise ("…which already replays deterministically" / "reproduce this run with Any.Reproducibly(seed, …)") and a qualified one ("…not reproducible from this seed alone"). The choice is driven by IsReproducible, which asks whether each operand draws from some source — but not whether the operands draw from the same source.

Combine computes (Any.cs):

RandomSource? source       = SourceOf(first) ?? SourceOf(second);          // the FIRST non-null source
bool          reproducible = IsReproducible(first) && IsReproducible(second); // AND of individual reproducibility

So two operands that each draw from a different seeded source are both "reproducible", reproducible is true, and the message names only the first source's seed — promising a deterministic full replay that seed cannot deliver.

var reference = Any.WithSeed(4242).String().StartingWith("ORD-"); // FixedRandomSource(4242)
var name      = Any.String().NonEmpty();                          // ambient source (seed B)
Any.Combine(reference, name, (r, n) => new Customer(n, OrderRef.Create(r))).Generate();
// composer throws → "… drawn from Any.WithSeed(4242), which already replays deterministically."
// FALSE: `name` came from the ambient source (B), not from WithSeed(4242); replaying 4242 reproduces only `reference`.

The defect is symmetric (reverse the order → it names the ambient seed and over-promises the same way), and applies to any mix of distinct seeded sources (two different Any.WithSeed(…), or WithSeed + ambient).

How it is reached in practice

Nobody writes that exact mix deliberately. It happens when a seeded operand is hidden behind a helper/fixture and combined with ambient generators, e.g.:

public static IAny<OrderRef> AnOrderRef() =>
    Any.WithSeed(4242).String().StartingWith("ORD-").As(OrderRef.Create); // WithSeed buried here

// elsewhere, believing everything is ambient:
Any.Combine(AnOrderRef(), Any.String().NonEmpty(), (r, n) => new Customer(n, r)).Generate();

Severity

The narrowest of the v1.0.0-audit findings: it needs the source mix and a composer failure, and the harm is a misleading replay hint on an already-failing test — not a silent green, not a wrong value. But it is the same class of defect as #312: the flagship diagnostic must not lie, and here it lies about reproducibility, the library's central promise.

Direction

The full promise should hold only when every operand draws from the same source instance the hint names; otherwise qualify it — exactly as for a foreign operand.

bool reproducible = DrawsOnlyFrom(first, source) && DrawsOnlyFrom(second, source);
// DrawsOnlyFrom(g, s) = IsReproducible(g) && ReferenceEquals(SourceOf(g), s)
  • Common good cases preserved: Combine(Any.Int32(), Any.String()) (same ambient singleton) and one shared Any.WithSeed(…) context stay full.
  • Bug fixed: distinct seeded sources → qualified ("not reproducible from this seed alone").
  • Accepted conservative edge: two separate Any.WithSeed(4242) (same seed, different instances) become qualified though technically deterministic — under-promising, which is safe, and an unusual pattern.

Acceptance criteria

  • A Combine over two different seeded sources no longer promises a full replay; existing ambient / shared-context / foreign cases are unchanged.
  • Example test in JustDummies.UnitTests (message content is the example suite's job, ADR-0040), confirmed red before the fix.
  • No public API change; suites stay green; 0 warnings.

Context

Surfaced by the 2026-07 JustDummies v1.0.0 readiness audit as problem #4. Sibling of #310/#312/#317.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions