Skip to content
Ameya Borkar edited this page Jun 10, 2026 · 1 revision

Replay — deterministic What-If

What-If Replay (throttlekit/testkit, @experimental) records a leaf limiter's decisions into a deterministic, JSON-serializable trace, then replays that trace — against the recorded policy (an identity self-check that proves bit-exact reproduction) or against a candidate what-if. It is the substrate beneath Policy Plans and the live Lens Replay tab.

The library — record, then replay

import { recordLimiter, replay } from "throttlekit/testkit";

// record a leaf limiter's synchronous decisions over real arrivals
const rec = recordLimiter({ strategy: "fixedWindow", limit: 3, windowMs: 1000 });
for (let i = 0; i < 6; i++) rec.limiter.checkSync("tenant-a");

// replay the SAME policy → an identity self-check (must reproduce bit-for-bit)…
replay(rec.trace, {});
// …or replay a single-field candidate what-if — "what would limit 2 have done?"
const diff = replay(rec.trace, { candidateField: { limit: 2 } });
Piece What it is
recordLimiter(spec) Wraps a leaf limiter; every checkSync appends to a deterministic ReplayTrace (the (now, key, cost) → Decision timeline)
replay(trace, opts) Re-runs the trace — identity self-check, or a candidateField what-if — and returns the divergence
set / scale / swap The candidate-compare DSL (one field set, a limit scaled, a strategy swapped)
scorecard / rankByFlips Score and rank many candidates over one trace
buildStrategy (from throttlekit/config) the single source of truth that rebuilds a Strategy from a LimiterSpec — the replay rebuilder reuses it, so a replayed limiter is the same limiter

Fail-loud throughout. A replay that can't be trusted refuses rather than guesses: a ReplayRefusedError carries a machine-readable reason, and a structural trust boundary validates any parsed or transmitted trace before replay trusts it.

Why it's deterministic

Replay rests on a cross-store-checked determinism substrate: the leaf algorithms are pure functions of (now, key, cost), and a conformance suite proves the Memory and Redis-Lua paths agree bit-for-bit over thousands of generated timelines. With an explicit now (a ManualClock), a recorded decision is reproducible exactly — which is what lets a what-if attribute a flip to the policy change, not to timing noise.

On the server — the Lens Replay tab

throttlekit-server can shadow live traffic for an in-terminal what-if. Enable a replay: block (opt-in, off by default — it records redacted keys), open the Replay tab in ThrottleKit Lens, and press r to replay a configured candidate over the traffic recorded so far:

replay:
  enabled: true
  policies: api, search          # leaf-rate policies to shadow (omit ⇒ all leaf-rate)
  maxSteps: 50000                # per-policy recording cap = the memory bound
  candidate: { policy: api, set: { limit: 200 } }

For each shadowed policy the server runs an isolated shadow of the live arrival stream through a cold, deterministic (ManualClock) copy of the limiter — a post-decision tail over its own store, so it can never change, delay, or break a production decision. It stops recording at maxSteps (a distinct-key flood can't exhaust memory; the trace is then honestly flagged truncated and the what-if refuses rather than understating). The pane shows the directional allow↔deny flip ledger, or an honest empty / truncated / refused state — never a faked number.

Honest scope

  • The flip count is candidate-spec vs the deterministic baseline over this traffic shapenot a replay of production's exact decisions (a Redis-backed or warm production node decides differently from a cold shadow).
  • Replay covers leaf-rate (and cost) limiters; concurrency / escrow / federated / joint-LP axes are not deterministically replayable — observe them live via binding-axis attribution.
  • The server Replay tab is a --tui feature (the what-if is a keybind); it is distinct from decision capture (Operations) — capture is the durable forensic record, replay is the in-memory deterministic what-if.

See also

Clone this wiki locally