Add ws cap liveness check#296
Merged
Merged
Conversation
KirillPamPam
force-pushed
the
head-liveness
branch
from
July 22, 2026 12:53
b9eb3f5 to
5cb641d
Compare
a10zn8
approved these changes
Jul 22, 2026
KirillPamPam
force-pushed
the
head-liveness
branch
from
July 23, 2026 08:38
cb15834 to
d2764d1
Compare
a10zn8
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Head-liveness gating of the WebSocket subscription capability (
WsCap)Summary
An EVM upstream whose head is driven by a WebSocket must now prove its head is live —
advancing consecutively — before it advertises the base WebSocket capability
WsCapand canback client subscriptions. A head that skips blocks or stalls, or a WebSocket disconnect, pulls
the upstream out of subscription serving until it recovers; its regular RPC routing is
untouched. The behavior is controlled by a new option,
disable-liveness-subscription-validation— its default is mode-dependent: validation isoff in
defaultmode and on instrictmode.Motivation
Ported conceptually from dshackle's
HeadLivenessValidator. Until nowWsCapwas asserted assoon as the WebSocket connector reported connected, so an upstream whose
newHeadssubscription was wedged or delivering gappy/stale heads could still be selected as a
subscription source and relay a bad stream to clients. This adds a liveness signal specific to
ws-driven heads.
Like dshackle, head liveness is deliberately not an availability/health signal: it feeds
only the subscription capability, never normal request routing.
WsCapgates subscriptionsonly — the
WsCapMatcheris added just for subscribe requests, and chain-levelSubMethodsadvertisement — so a not-live upstream still serves all ordinary RPC.
What changed
New head-liveness cap detector
WsHeadLivenessCapDetector(internal/upstreams/caps/ws_head_liveness_detector.go), genericover the cap it governs. It
selects over the WebSocket connector'sSubscribeStatesandthe shared head processor's
HeadEventstream, asserting the cap only while the connector isWsConnectedand the head is live.headLivenessTracker, ported from dshackle'sHeadLivenessValidatorImpl: ahead becomes live once it has produced 3 blocks in a row (2 consecutive height increments).
Following dshackle, any
diff <= 1counts as consecutive — duplicate heights and backwardreorgs are tolerated — and only a forward gap (
diff >= 2) resets the run and starts a5-minute cooldown, during which the head stays not-live even after it resumes advancing.
window (
measuredBlockTime × 3 × 2).measuredBlockTimestarts at the chain'sexpected-block-time, grows to the largest observed inter-block interval, and doubles on eachtimeout (capped at 10 min). A socket drop is still caught independently by the connector-state
gate, and the head processor's no-update watchdog still resubscribes.
Shared head processor
HeadProcessoris now built once per upstream (CreateHeadProcessor) and shared betweenthe head-event processor (which still owns its start/stop) and the cap detector (read-only
Subscribe).caps.DetectorInputgains aHeadfield carrying it.EVM wiring
EvmChainSpecificObject.CapDetectorsgatesWsCapwith the head-liveness detector when thehead connector is a WebSocket and validation is enabled; otherwise it keeps the plain
WsPresenceCapDetector. The chain'sexpected-block-timeis threaded into the detector toseed
measuredBlockTime. Poll-head EVM upstreams and non-EVM chains are unchanged, and theNewHeadsCap/LogsCap/PendingTxCapdetectors are untouched. EVM-only for now:consecutiveness assumes
+1height increments, which does not fit Solana's slot-based heads.Config option
disable-liveness-subscription-validation *boolonchains.Options(
yaml:"disable-liveness-subscription-validation"), settable inchains.yaml, inchain-defaults, or per upstream. Resolved (upstream → chain-defaults → chain settings) insetOptionsDefaultswith a mode-dependent fallback (trueindefaultmode,falseinstrictmode), plus a nil-safe accessorOptions.LivenessSubscriptionValidationDisabled().Config
trueskips the check and restores the historical "connected WebSocket ⇒WsCap" behavior.Left unset, it defaults to
trueindefaultmode (gate off) andfalseinstrictmode(gate on).
Notes / impact
subscriptions until its head has advanced 3 consecutive blocks — and, after any forward gap,
for a further 5-minute cooldown.
WsCapremoves the upstream as a target for all node-backedsubscriptions (and de-advertises sub topics if every upstream on the chain loses it); it never
affects plain RPC routing.
Docs
docs/nodecore/05-upstream-config.md— newdisable-liveness-subscription-validationoption.docs/nodecore/13-subscriptions.md— new "Head-liveness gate onWsCap" note.Testing
ws_head_liveness_detector_internal_test.go, fake clock): block-count path;duplicate heights and backward reorgs advance the run; forward-gap reset; cooldown (stays
not-live through a fresh run until the cooldown elapses);
measuredBlockTimegrowth;onTimeoutbackoff +
maxBlockTimecap; immediate recovery after a timeout.ws_head_liveness_detector_test.go): asserted only when connected ANDconsecutive; disconnect and forward-gap retract; backward-reorg and duplicate-while-live keep it;
a stalled head times out and retracts; nil connector / nil head never assert.
defaultandstrict) + updated full-configfixtures.
go build ./...,go test -race ./internal/..., andgolangci-lint runall pass.