You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ratesengine-ops {cctp,rozo,soroswap-skim,comet-liquidity,phoenix,blend}-backfill
subcommands (ADR-0029 §"SQL-backfill from soroban_events").
Complete per-source backfill set that re-feeds soroban_events
rows through the live Go decoders to populate per-source
hypertables — replacing the MinIO walks earlier decoder PRs
named as a follow-up. CCTP + Rozo are the simplest cases
(stateless decoders, one consumer.Event per source row, single
target table); Soroswap skim + Comet liquidity handle the
two-tuple topic shape (prefix in topic_0_sym, event kind in topic_1_xdr — byte-equality filter in the callback). Comet
also filters out swap-kind rows since they already populate trades via live ingest. Phoenix is the most complex: its
decoder buffers 3-5 events per action across four actions
(provide_liquidity / withdraw_liquidity / bond / unbond),
emitting LiquidityEvent or StakeEvent only when an instance's
field set is complete. Feeding events in
(ledger_close_time, ledger, tx_hash, op_index) order keeps the
buffer's age-based eviction quiet — orphans only fire on
genuinely-incomplete groups. Blend has the widest fan-out: 20
topics dispatch into three target tables (blend_positions,
blend_emissions, blend_admin). Auction events (the legacy
directional-price signal already covered by blend_auctions via
live ingest) are deliberately NOT backfilled by blend-backfill.
Supporting machinery: sorobanevents.Reconstruct(Row) rebuilds an events.Event
from a stored row (round-trip-tested vs Capture); Store.StreamSorobanEvents(ctx, from, to, contracts, topics, fn) is a Postgres-side filtered iterator; scval.DecodeScVecToArgs is the inverse of EncodeArgsAsScVec
(handles the op_args_xdr column → events.Event.OpArgs
conversion). All idempotent via the per-source table's existing ON CONFLICT DO NOTHING.
ledgerstream.Config.TolerateTrailingMissing (with companion TrailingMissingWindow). Closes the trailing-edge failure
that bit both verify-archive (project_62_diagnosis_2026_05_25)
and the 2026-05-26 soroban-events fill walk chunk 11
(ledger object containing sequence 62642880 is missing). When
the flag is set and the SDK reports a missing file within TrailingMissingWindow ledgers of the bounded to, Stream
returns nil (walk-complete) with a WARN. Mid-range gaps still
error — the window check guards against masking real corruption.
Default window 65536 (one Galexie 64k partition plus slack).
Wired into the standard LedgerstreamConfig helper (so ratesengine-ops backfill + the indexer's bounded preamble
inherit it), the verify-archive walker (the timer can now
re-enable), and the wasm-history walker. Delivery caveat
documented on the Config field: the SDK cancels its internal
context on a missing file, which can drop pre-fetched ledgers
in the buffer — operators relying on 100% coverage must clamp -to below the live tip, the tolerate flag is for graceful
exit at the trailing edge, not a substitute for tip-aware
range selection. Regression-tested in internal/ledgerstream/trailing_edge_internal_test.go (regex
parses every observed SDK wrap shape) and internal/ledgerstream/trailing_edge_stream_test.go (Stream
returns nil only when within-window; mid-range gaps + strict
mode still error).