Add model-based, property-based, mixed-version, and outage validation

[lan17]

Summary

Expand DialCache's validation beyond line/branch coverage with four complementary test classes:

1. property-based tests for parsers, policy overlays, keys, and payload envelopes;
2. model-based state-machine tests for cache serving and publication invariants;
3. mixed-version package/protocol compatibility tests; and
4. outage/resource scenarios that combine high cardinality, Redis degradation, source failures, stale candidates, compression, and observers.

The repository already has strong focused unit, integration, packed-package, liveness, and real Redis/Valkey coverage. This issue is about finding sequence-dependent and compatibility defects that example-based tests can miss, not increasing coverage percentages for their own sake.

Motivation

DialCache now has interacting dimensions including:

• request-local and process-local storage;
• two coalescing scopes and opt-out behavior;
• runtime policy overlays and deterministic ramps;
• tracked and untracked Redis modes;
• client-clock age enforcement and watermark fencing;
• source deadlines and Redis read deadlines;
• stale-on-error retained candidates;
• detached shadow validation and filling;
• compression envelopes and custom serializers;
• mixed ESM/CommonJS package entry points;
• rolling-upgrade compatibility constraints.

Most individual paths are well covered. The remaining risk increasingly lives in combinations and event orderings.

Phase 1: property-based tests

Add generated inputs around pure or nearly pure boundaries.

Runtime policy resolution

Generate default and sparse runtime overlays across:

• local/remote TTLs;
• ramps at boundaries and malformed values;
• request-local and coalescing booleans;
• stale-on-error inheritance and explicit disablement;
• remote read timeout inheritance;
• nested shadow policy.

Properties should include:

• omission preserves the default leaf;
• an explicit override replaces only that leaf;
• an explicit disable cannot be mistaken for omission;
• invalid runtime policy disables only its documented scope;
• merge/resolve is deterministic and does not mutate either input;
• static validation and runtime fail-open behavior agree on valid domains.

Reconcile the generated model with the layer-first shape if #144 lands.

Cache-key normalization

Generate valid and malformed scalar/object key shapes and assert:

• valid canonical keys are deterministic;
• argument order does not change identity;
• undefined arguments are omitted exactly;
• string-canonical scalar equality remains intentional;
• malformed JavaScript shapes fail open and cannot collide with legitimate values after Validate cache-key shapes before canonicalization #97;
• tracked value/watermark keys retain the required shared Redis hash tag.

Redis frame and compression envelopes

Generate arbitrary strings and Buffers, including every leading-byte pattern and boundary length, and assert:

• encode/decode round trips;
• compression/escape/decompression round trips;
• input Buffers are never mutated;
• repeated loads of retained payloads are independent;
• malformed, short, unsupported, future, and over-limit inputs follow their documented miss/error classifications;
• output limits hold without unbounded allocation.

Time boundaries

Generate creation times, fresh TTLs, stale maxima, watermarks, and reader times around exact boundaries:

• age < F is fresh;
• age === F is stale;
• age < M may be retained;
• age === M is unavailable;
• future frames fail closed;
• createdAtMs <= watermark is fenced.

Phase 2: model-based cache state machine

Build a small reference model that accepts bounded events such as:

enable / disable scope
request-local hit / miss
process-local hit / miss
remote hit / miss / retained / error / timeout
source success / rejection / timeout
remote write success / failure / late settlement
invalidation
shadow admission / timeout / supersession
clock advancement or skew

Generate short event sequences and compare the implementation against the model.

Core invariants should include:

• a fenced tracked frame is never served as an ordinary hit;
• cache plumbing never replaces the authoritative source rejection;
• stale recovery uses only the initially retained snapshot and never rereads Redis;
• tracked invocations that reach the Redis path obey the exact local-publication suppression rules;
• request-local memoization remains an outer-scope snapshot;
• coalesced followers share the leader's documented result and budget;
• coalesce: false gives callers independent work;
• each semantic read produces exactly one request/get trail and at most one miss;
• detached shadow work never affects the caller result;
• every admitted background flight eventually releases capacity after underlying work settles;
• observer failures never affect correctness.

Use deterministic fake clients and clocks; do not make the model suite depend on wall-clock sleeps.

Phase 3: mixed-version compatibility matrix

Automate compatibility checks across current and selected previous package versions or frozen historical fixtures.

Representative matrix:

current reader / current writer
current reader / previous writer
previous reader / current writer
current node-redis adapter / previous frame fixture
current GLIDE adapter / previous frame fixture

Cover at least:

• tracked and untracked values;
• compressed and uncompressed payloads;
• stale-retention disabled and enabled where the older version supports it;
• current/legacy custom semantic Redis clients;
• packed ESM and CommonJS roots/subpaths;
• documented error identity across entry points;
• metric-schema changes that require mixed-fleet query guidance.

The suite should distinguish:

• wire-compatible rolling upgrades;
• source/type compatibility breaks;
• intentionally unsafe overlapping protocols that require drain/purge/cutover;
• rollback-safe versus rollback-unsafe transitions.

Do not claim compatibility where the release documentation requires an operational cutover. Encode that incompatibility as an expected result.

Phase 4: outage and resource scenarios

Add repeatable scenarios, aligned with the artifact/environment work in #35, combining:

• high-cardinality distinct keys;
• same-key coalesced fan-out;
• Redis latency, rejection, disconnect, and late settlement;
• source timeout/outage;
• stale-on-error candidates;
• compressible and incompressible payloads;
• metrics and logging enabled;
• tracked and untracked modes.

Measure and assert where applicable:

• source call count and maximum concurrency;
• Redis command count;
• retained external/heap memory while stale candidates are live;
• release to baseline after completion/timeout;
• event-loop delay during compression/decompression;
• detached-flight and process-flight state returning to zero;
• bounded log volume if log sampling is introduced;
• no unhandled rejection or process-liveness leak.

Timing values remain informational unless a stable environment-specific budget is explicitly approved. Semantic and resource-release assertions belong in CI where deterministic.

Implementation guidance

• Prefer built-in test generation or a small established property-testing dependency only if it materially reduces custom harness code.
• Record failing seeds and make them directly replayable.
• Keep generated case counts bounded in normal CI; allow a larger maintainer/nightly mode.
• Shrink failures to a minimal reproducible sequence.
• Reuse existing fake Redis clients, fake clocks, integration fixtures, and packed-package harnesses.
• Keep heavy performance artifact generation in Add a repeatable performance and scale benchmark suite #35; this issue owns correctness properties and compatibility expectations.

Acceptance criteria

• Property tests cover runtime policy overlays, key normalization, frame/compression round trips, and exact time boundaries.
• Failing generated cases print a stable replayable seed and a minimized input or event sequence.
• A reference state machine validates serving, fallback, publication, coalescing, stale-recovery, shadow, and observer invariants across generated orderings.
• Mixed-version tests exercise selected real packed versions or frozen fixtures in both ESM and CommonJS.
• Every version pair is classified as compatible, expected source break, or operationally incompatible; unsafe combinations are not silently skipped.
• Outage scenarios cover high-cardinality and same-key traffic with Redis and source degradation together.
• Resource tests prove stale payloads, shadow frames, timers, and flight state are released after settlement or timeout.
• Compression scenarios record event-loop delay and threadpool/CPU effects across representative payloads without imposing noisy universal timing gates.
• Normal CI remains deterministic and reasonably bounded; deeper generated runs have a documented maintainer command.
• Existing focused tests remain; generated/model suites supplement rather than replace readable regression cases.

Non-goals

• Replacing current unit/integration tests with opaque generated tests.
• Universal latency thresholds across GitHub-hosted runners.
• Claiming compatibility across explicitly incompatible historical protocols.
• Adding production runtime assertions solely to make the test model easier.

Related

• Add a repeatable performance and scale benchmark suite #35 — repeatable performance and scale benchmark suite and machine-readable artifacts.
• Characterize and guard watermark cardinality and Redis Cluster hot slots #42 — watermark cardinality and Redis Cluster hot-slot evidence.
• Establish a local-hit performance budget and remove avoidable hot-path work #43 — local-hit performance budget.
• Validate cache-key shapes before canonicalization #97 — malformed runtime key validation.
• Refactor cache config to group policy by layer #144 — layer-first config shape.