Skip to content

feat(relayer): persist eth scan progress#344

Merged
sadiq1971 merged 4 commits into
mainfrom
feat/relayer-persist-scan-progress
Jul 2, 2026
Merged

feat(relayer): persist eth scan progress#344
sadiq1971 merged 4 commits into
mainfrom
feat/relayer-persist-scan-progress

Conversation

@sadiq1971

@sadiq1971 sadiq1971 commented Jul 2, 2026

Copy link
Copy Markdown
Member

Problem

The relayer only persisted its Ethereum scan offset as a side effect of processing a deposit. Ranges with no deposits advanced the in-memory scan position but never wrote to chain_state, so on restart the poller resumed from the last block that happened to contain a deposit — re-scanning potentially large empty ranges (or, if eth_start_block was bumped manually to compensate, risking skipped blocks).

Fix

The poller now emits a scan-progress checkpoint after every fully scanned slice — including slices with no deposits — and the DB offset is persisted from it in real time.

  • After each successful eth_getLogs slice, WatchDepositEvents calls the existing handler with a DepositEvent{Checkpoint: true} carrying the last scanned block. Progress advances only through the last successful slice; a failing slice and everything after it is retried next tick.
  • The Ethereum source turns a checkpoint event into a lightweight Checkpoint relayer.Event that rides the same ordered channel behind that slice's deposits.
  • The processor persists the offset for a checkpoint and does no transfer work; persistOffset dedupes against the last saved offset.

Carrying the signal on the existing handler (rather than a new onProgress callback) keeps the EthereumBridgeClient interface — and its generated mock — unchanged, so the diff stays contained.

Nothing missed / nothing double-processed

  • The checkpoint for block N is consumed in order, after every deposit ≤ N — the processor loop is single-threaded — so the offset never advances past an unprocessed deposit.
  • Each deposit is durably recorded via CreateTransfer (idempotent, ON CONFLICT DO NOTHING) before submission is attempted. A failed submit stays pending and is retried by the reconcile loop, independent of scan progress — so advancing the scan offset past it can't drop it.
  • On restart the offset is a block that was fully scanned; the poller resumes at offset+1, and any re-seen deposit is deduped by CreateTransfer.

Tests

  • TestProcessor_Start_CheckpointPersistsOffsetWithoutProcessing — a checkpoint event persists the offset and runs no transfer processing.
  • TestEthereumSource_StreamEvents_EmitsScanCheckpoint — a checkpoint DepositEvent from the handler becomes a checkpoint relayer.Event carrying the scanned block.
  • Existing deposit-mapping / engine tests unchanged (interface signature preserved).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a checkpointing mechanism to track and persist scan progress during Ethereum deposit event polling. It updates WatchDepositEvents to accept an onProgress callback, which emits in-order checkpoint events that the processor handles by persisting the offset without triggering transfer processing. A critical logic error was identified in pkg/ethereum/client.go where currentBlock is updated before onProgress successfully executes. If onProgress fails, the poller would skip re-scanning the failed block range on the next tick. It is recommended to only update the block tracking state after onProgress successfully completes.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread pkg/ethereum/client.go Outdated
@codecov-commenter

codecov-commenter commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 5 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@b20e2dc). Learn more about missing BASE report.

Files with missing lines Patch % Lines
pkg/ethereum/client.go 0.00% 3 Missing ⚠️
pkg/relayer/engine/source.go 85.71% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #344   +/-   ##
=======================================
  Coverage        ?   31.75%           
=======================================
  Files           ?      154           
  Lines           ?    11987           
  Branches        ?        0           
=======================================
  Hits            ?     3806           
  Misses          ?     7888           
  Partials        ?      293           
Flag Coverage Δ
unittests 31.75% <75.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/relayer/engine/processor.go 80.35% <100.00%> (ø)
pkg/relayer/engine/source.go 95.55% <85.71%> (ø)
pkg/ethereum/client.go 6.13% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sadiq1971 sadiq1971 force-pushed the feat/relayer-persist-scan-progress branch from 68a4954 to 935e509 Compare July 2, 2026 11:18
The relayer only persisted its Ethereum scan offset as a side effect of
processing a deposit, so block ranges with no deposits never advanced the
persisted offset — on restart the poller re-scanned from the last block that
happened to contain a deposit.

The poller now emits a scan-progress checkpoint after every fully scanned
slice (including empty ones), carried on the existing handler as a
DepositEvent{Checkpoint: true}. The Ethereum source turns it into a checkpoint
relayer.Event that rides the same in-order channel behind that slice's
deposits; the processor persists the offset from it and does no transfer work.

Because the checkpoint is delivered in order after the slice's deposits — and
each deposit is durably recorded via CreateTransfer (idempotent) before submit —
the offset never advances past an unprocessed deposit, so nothing is missed or
double-processed. Carrying the signal on the existing handler keeps the
EthereumBridgeClient interface (and its mock) unchanged.
@sadiq1971 sadiq1971 force-pushed the feat/relayer-persist-scan-progress branch from 935e509 to fef4ffa Compare July 2, 2026 11:27
sadiq1971 added 3 commits July 2, 2026 17:38
Move currentBlock/setLastScannedBlock past the checkpoint handler call so a
failed checkpoint re-scans and re-checkpoints the same range on the next tick
instead of skipping its watermark, matching the documented intent.

@dhyaniarun1993 dhyaniarun1993 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I encountered this bug while testing prompt token integration. Thanks for looking into this.

@sadiq1971 sadiq1971 self-assigned this Jul 2, 2026
@sadiq1971 sadiq1971 merged commit 123a84a into main Jul 2, 2026
4 checks passed
@sadiq1971 sadiq1971 deleted the feat/relayer-persist-scan-progress branch July 2, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants