fix(client): reconnect policy — terminal 4xxx closes, jittered backoff (ADR-0016)

[grrowl]

Fixes #25
Fixes #26

Problem

The transport auto-reconnected from two sites (socket close handler, connect-failure catch) on a fixed reconnectDelayMs, blind to CloseEvent.code:

• a server accept-then-close(4403) auth rejection — the only client-readable rejection pattern for a browser WebSocket — retried forever (Transport auto-reconnect ignores application close codes (4xxx) — terminal auth rejections retry forever #25)
• a DO outage produced a fixed-interval thundering herd (Auto-reconnect uses a fixed delay — add exponential backoff + jitter (or a reconnect policy hook) #26)

Design (ADR-0016)

• Policy function over config flags. One new option subsumes the space: reconnectDelay?: (attempt, closeCode?, closeReason?) => number | null (null = stop). Both reconnect sites route through a single scheduleReconnect.
• Default: jittered capped backoff, terminal 4xxx. defaultReconnectDelay gives full-jitter exponential backoff (base = reconnectDelayMs, default 250 ms; cap 30 s, never below the base; attempt counter resets on a successful open) and returns null for application close codes 4000-4999 — the server closed on purpose, retrying cannot succeed.
• Terminal closes are observable. New onClosed(code, reason) hook fires exactly when the policy returns null (connect failures report undefined — no close frame), so auth closes are distinguishable from transient drops. Replaces the open()-listener workaround.
• Invariant preserved: reconnecting is still set at SCHEDULING time (demand-driven connect resubscribe window, ADR-0011 grill), and also on a terminal stop so an app-driven connect() after re-auth still resubscribes from the cursor.

reconnectDelayMs keeps working as the default policy's base. Pre-existing reconnect suites (reconnect, reconnect-recovery, reconnect-window) pass unchanged.

Adversarial review (codex gpt-5.5)

• Caught (fixed): a stale reconnect timer from a transient drop could fire after a later terminal 4xxx close, retrying past "stop" and double-firing onClosed; it also survived close(). The single pending timer handle is now tracked and cancelled on successful open, terminal stop, close(), and supersession — regression tests added.
• Caught (fixed): a stale socket's late close event (after close() + immediate connect()) could null the live socket; close handling is now socket-instance guarded.
• Rebutted: "terminal stop should reject pending tx/fetch/seq waiters immediately" — waiters were never rejected on any unexpected close (only on intentional close()); they fail loud via their existing timeoutMs. Unchanged pre-existing semantics, noted as a possible follow-up.

Tests

npx vitest run: 46 files, 209 tests, all green. New suites: tests/reconnect-policy.test.ts (policy unit tests + workerd terminal/transient/attempt-counter/stale-timer coverage), tests/auth-path.test.ts (upgrade-status vs application-close auth surfaces, incl. transport-vs-4403 end to end).

🤖 Generated with Claude Code

[grrowl]

Post-PR codex (gpt-5.5) adversarial pass on the final diff found one HIGH: close() while a reconnect-initiated open() was in flight would still install the resolved socket — a live, resubscribed connection after teardown. The timer-cancellation added earlier only covered close() before the timer fired.

Fixed in 3e8f731 with a closeEpoch captured before await open() and checked after: a mid-flight close() invalidates the attempt and the orphan socket is closed instead of installed. An epoch (not an intentionallyClosed check after the await) because explicit connect() after close() is pinned, allowed behavior. Red-first regression test included; full suite 45 files / 210 tests green.

The same pass explicitly cleared the previously-fixed paths: no stale-timer resurrection in the transient-drop → demand-driven-connect → terminal-close path, resubscribe invariant preserved, attempt counter grows/resets correctly.

🤖 Generated with Claude Code