Skip to content

feat: add fallback deadlines and coalescing state#69

Merged
lan17 merged 4 commits into
mainfrom
codex/issue-41-single-flight-lifetimes
Jul 20, 2026
Merged

feat: add fallback deadlines and coalescing state#69
lan17 merged 4 commits into
mainfrom
codex/issue-41-single-flight-lifetimes

Conversation

@lan17

@lan17 lan17 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Closes #41

Summary

This change gives enabled cache fallbacks a finite lifetime and makes process-scoped single-flight health exactly inspectable, without unsafe leader eviction or a registry-only global cap.

Enabled fallbacks use a 60-second monotonic deadline by default. A cached wrapper can select a positive fallbackTimeoutMs or use null to disable the guard. Followers share their leader's remaining budget. Timeout rejects the DialCache chain with FallbackTimeoutError, clears the flight through normal settlement, and prevents the late fallback result from reaching serializer, Redis, or local-cache publication. It does not cancel the underlying operation.

API and semantics

fallbackTimeoutMs is validated when cached() creates the wrapper. It accepts positive safe integer milliseconds through 2,147,483,647, plus explicit null. Invalid options do not reserve the use-case name.

  • The timer starts only when a call that began enabled reaches fallback. Calls that began outside an enabled context remain pass-through.
  • One referenced timer belongs to each fallback leader; process and request-local followers reuse the leader promise and timer.
  • The timer rounds delays up, checks the monotonic deadline when invoked, and rearms after an early callback.
  • Synchronous fallback work counts against the deadline, but JavaScript cannot preempt synchronous work or event-loop starvation. Rejection is delivered after control returns, and DialCache checks elapsed monotonic time before accepting a result.
  • Keeping the timer referenced guarantees delivery when it is the only active handle. Abandoned enabled work can therefore keep a short-lived process alive until its deadline; shutdown code should drain outstanding work.
  • Early settlement clears the timer. Late settlement remains observed to prevent unhandled rejections, but a timed-out result cannot continue to DialCache publication.
  • This is a wait-and-publication boundary, not cancellation. Source-native timeouts or AbortSignal support remain preferred.

Liveness boundary

fallbackTimeoutMs governs only fallback execution. DialCache does not claim to cancel or time-bound Redis, serializers, configuration providers, ramp samplers, or other injected async work. Public JSDoc and the README require every injected boundary to settle within an application-defined budget.

Valkey GLIDE requestTimeout bounds client waiting, not server-side cancellation. node-redis 4.7 connection and queue settings do not bound a dispatched response; its AbortSignal can remove only work still waiting to be sent. Applications needing strict finite settlement must provide a semantic-client policy, including ambiguous-write handling.

No Redis key, frame, Lua protocol, invalidation, serializer, or cached-value contract changes.

Coalescing observability

getCoalescingState() returns a detached snapshot for this DialCache instance:

{
  process: { activeLeaders, activeFollowers, oldestLeaderAgeMs },
}

Process flights are registered before synchronous leader work, carry a monotonic start time, and maintain exact follower counts. Joins, cleanup, and oldest-flight lookup remain O(1). Request-local flights remain lifecycle-owned by the outer enable() scope and are intentionally excluded.

There is no automatic leader replacement or global process-flight cap. Eviction can permit duplicate late writes, while a metadata cap does not bound backend work. Application admission control owns distinct-key concurrency; the snapshot shows whether finite resource budgets are working.

Compatibility and performance

The finite default deliberately changes enabled fallbacks that previously remained pending beyond 60 seconds. fallbackTimeoutMs: null is the explicit migration escape hatch.

Cache hits allocate no fallback timer, and followers reuse the leader promise and timer. Tracking adds one monotonic clock read and one small record per process leader; pre-work registration adds no promise, timer, or microtask. A Node 22.22 base-versus-head probe over 200,000 calls measured about 0.29 microseconds per immediate enabled fallback and 0.10 microseconds per process-local hit. These timings are informational; benchmark correctness assertions are deterministic.

Validation

  • corepack pnpm check: typecheck, 211 unit tests with coverage gates, build/declarations, packed ESM/CJS consumers
  • Package probes on Node 22.22 and 20.17, including real ESM/CJS subprocesses where the fallback timer is the only active handle
  • 46 Redis/Valkey/cluster integration tests
  • Five benchmark scenarios at 100,000 iterations and 5,000-way fan-out
  • Negative boundary coverage for pending configuration, ramp sampler, Redis read/write, and serializer work
  • Exact-head CI: https://github.com/lan17/DialCache/actions/runs/29709225990
  • Fresh discovery found two P2 and four P3 issues; all were fixed. A post-fix pass found one P3 documentation issue; it was fixed. The final independent exact-head pass returned no actionable P0-P3 findings.

@lan17
lan17 marked this pull request as ready for review July 20, 2026 00:03
@lan17
lan17 merged commit 34a45fa into main Jul 20, 2026
2 of 5 checks passed
@lan17
lan17 deleted the codex/issue-41-single-flight-lifetimes branch July 20, 2026 00:45
lan17 added a commit that referenced this pull request Jul 20, 2026
Combines runtime config overlays with the fallback deadline and
coalescing state work from #69. Both cached() registration validations
(defaultConfig snapshot and fallbackTimeoutMs) run before the use case
name is registered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Require finite single-flight lifetimes and expose observability

1 participant