Skip to content

feat(runtime): persist a per-subscription log cursor for gap-free resume across restarts#301

Merged
mfw78 merged 2 commits into
developfrom
feat/40-log-cursor-resume
Jul 9, 2026
Merged

feat(runtime): persist a per-subscription log cursor for gap-free resume across restarts#301
mfw78 merged 2 commits into
developfrom
feat/40-log-cursor-resume

Conversation

@mfw78

@mfw78 mfw78 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What

Makes chain-log delivery gap-free across engine restarts, not just within a process. A module opts a subscription in with one manifest flag:

[[subscription]]
kind = "chain-log"
chain_id = 11155111
address = "0x..."
event_signature = "0x..."
resume = true

The host then persists a durable cursor (the last dispatched block for that subscription) and, on boot, re-opens the watch_canonical_logs_from poller from that block instead of at the current head, clamped by MAX_SYNC_BACK_BLOCKS.

Why

Closes #300. #299 made log delivery gap-free within a process (the canonical poller + retry layer + sync-back-on-reopen). Across a process restart the poller still opened at head, dropping logs mined during downtime. This closes that gap for subscriptions that want it.

Design (host owns the cursor, module owns idempotency)

  • Host-owned cursor. Keyed by keccak(chain_id | address | topic) under the module's own store namespace, alongside the existing last_dispatched_block block marker. It is not derived from the alloy Filter: Filter's Hash uses a process-randomized HashSet, so it is not reproducible across restarts. The module never manages a cursor or a boot handshake.
  • Written after dispatch, not on send. The reconnect task's last_seen_block advances when it enqueues a log, but a crash with logs still buffered would record un-dispatched blocks as done. So the supervisor writes the cursor only after a successful dispatch_chain_log, mirroring the block marker. Single writer (supervisor), one-shot boot read (supervisor); the spawned task never touches the store.
  • At-least-once. Boot resumes at the cursor block and replays it in full (a mid-block crash pre-restart loses nothing); redelivery is deduped by the module's chassis idempotency journal. Exactly-once (a transactional cursor + module-state commit) is explicitly out of scope.
  • Default off. resume = false reproduces today's behaviour exactly (initial_cursor = None -> poller starts at head, no history replay).

Testing

cargo clippy --workspace --all-targets --all-features --locked -- -D warnings clean, cargo test -p nexum-runtime (206 passed, including chainlog_cursor_key stability/uniqueness tests), full cargo build --workspace, fmt, no em-dashes, all under the pinned 1.94 toolchain.

Depends on

#299 (the canonical poller, poller_resume_block, and MAX_SYNC_BACK_BLOCKS). This PR is stacked on it.

AI Assistance

Claude Code used for the seam mapping, the implementation, and this description.

@mfw78 mfw78 force-pushed the feat/40-log-cursor-resume branch from 5c65ca3 to 4568808 Compare July 9, 2026 06:13
Base automatically changed from feat/40-canonical-log-poller to develop July 9, 2026 06:45
mfw78 added 2 commits July 9, 2026 06:49
A `[[subscription]] resume = true` chain-log subscription now survives an
engine restart: the host persists the last dispatched block per
subscription and re-opens the poller from there (clamped by
`MAX_SYNC_BACK_BLOCKS`) instead of at head, so logs mined while the process
was down are re-delivered rather than lost.

- Manifest gains `resume: bool` (default false) on the chain-log
  subscription; whether a subscription needs gap-free resume is
  module-specific, so it is opt-in and declarative.
- The cursor is host-owned, keyed by `keccak(chain_id | address | topic)`
  in the module's store namespace - not the alloy `Filter`, whose hash
  uses a process-randomized `HashSet` and is not reproducible across
  restarts. It is read once at boot and written by the supervisor only
  after a successful dispatch (never on the task's buffered send), so a
  block is never recorded as done before the module processed it.
- Delivery is at-least-once: the boot resume replays the cursor block in
  full, deduped by the module's chassis idempotency journal. Exactly-once
  (a transactional cursor + state commit) is out of scope.

Closes #300
@mfw78 mfw78 force-pushed the feat/40-log-cursor-resume branch from 4568808 to 2dec80c Compare July 9, 2026 06:50
@mfw78 mfw78 merged commit d4bab4a into develop Jul 9, 2026
11 checks passed
@mfw78 mfw78 deleted the feat/40-log-cursor-resume branch July 9, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtime: persist a per-subscription log cursor for gap-free resume across restarts

1 participant