Rates Engine v0.5.0-rc.61

[v0.5.0-rc.61] — 2026-05-20

Fixed

• entries counter expanded to ALL observer-driven sinks
  (follow-up to the blend/router/defindex fix). Same session,
  same root cause: the seed query only knew about trades +
  oracle_updates, leaving the supply observers' tables
  (account_observations, trustline_observations,
  claimable_observations, lp_reserve_observations,
  sac_balance_observations, sep41_supply_events) silently
  excluded from per-source entries even though they're the
  primary observable activity surface for those sources. Now:
  (1) SeedSourceEntryCounts UNIONs all six observer tables
  with literal source names matching each observer's
  SourceName constant; (2) each persister
  (persistAccountObservation, persistTrustlineObservation,
  …, persistSEP41SupplyEvent) calls bumpEntryCount after a
  successful insert so the steady-state counter stays current
  between seed reconciliations. The result: /v1/diagnostics/ingestion
  surfaces entries for accounts, trustlines,
  claimable_balances, liquidity_pools, sac_balances,
  sep41_supply alongside the trade + oracle sources — the
  full "total decoded protocol activity" the user asked for.
• entries counter now tracks total protocol activity, not just
  trades. User-reported: /v1/diagnostics/ingestion showed 0
  entries for blend (writes to blend_auctions, not trades),
  defindex (Phase A log-only sink — no storage table),
  soroswap-router (same — log-only). Root cause:
  SeedSourceEntryCounts only UNION-ALL'd over trades +
  oracle_updates. Three-part fix:
  (1) SeedSourceEntryCounts query extended to also UNION
  blend_auctions (literal source 'blend') and fx_quotes
  (its nullable source column COALESCE'd to 'unknown-fx' so
  unlabelled rows still surface rather than vanish). (2) New
  Store.BumpSourceEntryCount(ctx, source, n) method — single
  UPSERT with ON CONFLICT DO UPDATE SET entry_count = entry_count + EXCLUDED.entry_count. Cheap enough for per-event
  use on low-volume sinks. (3) sink.go wires the bump into
  every Phase A log-only case (soroswap-router, defindex) and
  every blend-auction persister (new / fill / delete). Shared
  helper bumpEntryCount logs failures at Warn — bump errors
  don't fail the underlying decode (operator's
  ratesengine-ops seed-entry-counts reconciles drift). Other
  observer-driven sinks (account_observations,
  classic_supply_*, sep41_supply_events) are surfaced via the
  supply-observer pages per ADR-0023, not source-attributed
  entries — documented in the updated seed-query comment.

Added

• internal/sources/cctp decoder for Circle CCTP v2 events
  (#40 Phase 1). Pure-function decoder package for all four CCTP
  events: DepositForBurn (outbound USDC burn from
  TokenMessengerMinter), MintAndWithdraw (inbound mint after
  attestation), MessageSent (wire envelope, paired with
  DepositForBurn), MessageReceived (wire envelope, paired with
  MintAndWithdraw). events.go defines the four canonical Go
  types with full BytesN<32>-as-hex serialisation for the
  cross-chain address fields (mint_recipient,
  destination_token_messenger, destination_caller, nonce,
  sender); decode.go exposes Classify + four Decode*
  functions with explicit ErrMalformedTopic /
  ErrMalformedBody sentinels for schema-drift detection;
  decode_test.go covers 16 cases including all four event
  types' happy paths, ADR-0003 large-i128 round-trip on
  DepositForBurn's amount, short-topic + missing-body-field
  drift signals, MessageSent's dual ScMap/raw-Bytes paths
  (forward-compat against macro layout shifts), and topic-symbol
  encoding stability for all four. Per ADR-0013 the decoder
  doesn't import xdr directly — uses inferred-type entries
  through scval returns, same pattern as Soroswap.
  NOT yet wired — no registry entry, no consumer.Source,
  no migration. Wiring follows the storage-shape decision
  (bridge_events shared with Rozo vs cctp_events separate)
  per docs/architecture/cctp-stellar-coverage.md §Storage.
  CCTP + Rozo decoders shipping in parallel means the storage
  layer can be designed against both event shapes at once.
• internal/sources/rozo decoder for Rozo v1 Payment events
  (#41 Phase 1). Pure-function decoder package — events.go
  defines the canonical Payment + Flush Go types and the
  pre-encoded topic-symbol constants; decode.go exposes
  Classify, DecodePayment, DecodeFlush with explicit
  ErrMalformedBody for field-missing surfacing; decode_test.go
  carries 12 parallel tests including the ADR-0003 large-i128
  round-trip (locks the *big.Int → string precision invariant
  against the int64-truncation bug class) and topic-symbol
  encoding stability guards (re-encoded bytes must match the
  package-init constants — drift would silently break
  Classify). The package is NOT yet wired — no
  registration in internal/sources/external/registry.go, no
  consumer.Source impl, no dispatcher_adapter.go. Wiring +
  storage layer follows the bridge_events vs rozo_events
  shape decision (operator-gated; see
  docs/architecture/rozo-stellar-coverage.md §Storage).
  Capturing the decoder logic in code with tests means the
  implementation phase doesn't have to re-derive the on-chain
  event schema from the contract source — it's the
  smallest-possible-PR that advances #41 without committing to a
  specific storage shape.
• ClassBridge source class for cross-chain transfer protocols
  (#40 + #41 unblock). Adds ClassBridge Class = "bridge" to
  internal/sources/external/framework.go alongside the existing
  six classes. Bridges (Circle CCTP, Rozo) move tokens between
  chains rather than exchanging them at a price — a
  deposit_for_burn on Stellar + mint_and_withdraw on Ethereum
  is one logical USDC transfer, not a two-leg trade. Excluded
  from VWAP by default (IncludeInVWAP: false); reported
  alongside for cross-chain flow attribution and as the
  cross-chain side of Algorithm 3 supply accounting (complements
  ADR-0023's SEP-41 supply observer that already tracks
  classic trustline-driven mints/burns). The
  TestRegistry_ClassPolicy invariant ("only ClassExchange may
  VWAP-contribute") covers ClassBridge unchanged.
  TestClassBridge_Defined locks the wire value so a downstream
  rename surfaces as a build break here rather than as a silent
  classification miss. Removes the primary operator gate from
  the #40 / #41 design docs — implementation can now proceed on
  the storage-shape decision alone (bridge_events shared vs
  per-protocol tables).

Docs

• docs/architecture/rozo-stellar-coverage.md — Rozo intents
  decoder + storage design (#41 design pass). Captures three
  distinct contract variants discovered in
  RozoAI/rozo-intents-contracts: (1) v1 Payment LIVE on
  mainnet at CAC5SKP5FJT2ZZ7YLV4UCOM6Z5SQCCVPZWHLLLVQNQG2RWWOOSP3IYRL
  (verified via StellarExpert) — emits PaymentEvent { from, destination, amount, memo } on ("payment", from) topic and
  FlushEvent { token, destination, amount } on ("flush",)
  topic; (2) v2 Forwarder + IntentBridge pre-mainnet with
  topic shapes ("forward", sender), ("memo_set",),
  ("created",), ("filled",), ("refunded",); (3) a newer
  rozo-intents package emitting ("intent_created", intent_id: BytesN<32>) + intent_filled / intent_failed /
  intent_refunded long-form symbols (status unclear — possibly
  v2.1 or v3 unifying variant). Shares CCTP's
  ClassBridge-or-not design question + the storage shape
  question (bridge_events shared with CCTP vs rozo_events
  separate). Recommends three-phase rollout: ship v1 Payment
  decoder now (the only live contract — user direction was "v1
  and v2", but v2 isn't deployed yet so Phase 1 covers reality),
  Phase 2 for v2 contracts when they deploy to mainnet, Phase 3
  for the rozo-intents variant after schema-status
  clarification. Implementation gated on the same operator
  decisions as CCTP.
• docs/architecture/cctp-stellar-coverage.md — CCTP-Stellar
  decoder + storage design (#40 design pass). Captures the
  three mainnet contract addresses (TokenMessengerMinter,
  MessageTransmitter, CctpForwarder), the four canonical event
  schemas (DepositForBurn, MintAndWithdraw, MessageSent,
  MessageReceived) extracted verbatim from
  circlefin/stellar-cctp/contracts/{token-messenger-minter-v2, message-transmitter-v2}/src/lib.rs, decoder strategy
  recommendation (Option A: topic-based via existing dispatcher,
  same pattern as Soroswap / Phoenix / Aquarius), and storage
  shape recommendation (new cctp_events hypertable via
  migration 0037 — bridge events don't fit trades). Surfaces
  five operator-gated design questions, primary being whether
  CCTP warrants a new ClassBridge source class (it doesn't fit
  ClassExchange — no trades, no price signal — or the existing
  ClassRouter semantic, which elides the cross-chain
  dimension). Implementation lands after class-design sign-off
  (per CLAUDE.md "Add a new on-chain Soroban DEX" + WASM-history
  walk before BackfillSafe: true). The user direction was
  "CCTP shouldn't have any history because it is brand new" —
  so initial implementation is live-only ingest from current
  ledger forward.

Changed

• Issuer-filter pushdown into listCoinsBaseSelect CTEs (#27).
  Live r1 EXPLAIN ANALYZE on /v1/assets?issuer=GA5Z…:
  per_asset_24h_vol's Partial HashAggregate scanned 256,724
  rows of prices_1m to materialise stats for every asset, then
  the outer SELECT discarded all but 9 (the actual issuer's
  asset count). 1.3M shared-buffer hits for a single-issuer
  query. The PostgreSQL 12+ default of inlining CTEs doesn't
  help here because each per-asset CTE has an aggregate
  (SUM/DISTINCT ON) that becomes a predicate-pushdown barrier;
  the issuer filter on ca.issuer_g_strkey is unrelated to the
  CTEs' GROUP BY asset_id, so the planner can't push through.
  Fix: when issuer is set in buildCoinsQuery,
  listCoinsBaseSelectSQL prepends a chosen_assets CTE that
  materialises the issuer's asset_id set once, and each of the
  nine per-asset CTEs adds AND base_asset IN (SELECT asset_id FROM chosen_assets) (per_asset_24h_vol also adds the
  symmetric quote_asset IN to the union's quote-side branch).
  The four xlm_usd CTEs deliberately stay unfiltered (they
  look up XLM specifically, not the caller's asset). Sentinel
  comments (/*PUSHDOWN_BASE*/, /*PUSHDOWN_QUOTE*/) embedded
  in the SQL get replaced in-place — keeping the const + the
  renderer adjacent rather than maintaining two parallel
  300-line SQL strings. q-search pushdown intentionally deferred
  — LIKE patterns on three columns combined with the outer
  LIKE rules don't reduce as predictably; if profiling later
  shows that path is hot, a q-side chosen_assets variant can
  be added. Backfill behaviour unchanged: unfiltered LIST
  (the dominant traffic pattern) is byte-for-byte the same SQL.
  Six tests cover the renderer + buildCoinsQuery branches
  (no-pushdown, with-pushdown, no-issuer, issuer-only,
  q-only-no-pushdown, issuer+q).

Fixed

• Prewarm extended to verified-currency canonical asset_ids
  (#37 follow-up). Real measurement on r1 post-rc.60:
  /v1/assets/usdc (slug form) was 144ms but
  /v1/assets/USDC-GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN
  (canonical form) was 3.3s — same underlying
  getCoinBySlugSQL, but the canonical form missed cache because
  prewarmLight only warmed ListCoinsExt + GetNativeCoinRow.
  Programmatic clients + the explorer's drill-out paths navigate
  by canonical asset_id, so this was the dominant user-visible
  slowness post-rc.59's native-only prewarm. Fix: catalogue load
  moved BEFORE the prewarm goroutine; prewarmCaches +
  prewarmLight now take a []string verifiedAssetIDs extracted
  from the catalogue (each Stellar-network entry's AssetID,
  excluding native + empty). Each entry feeds a
  coins.GetCoinByAssetID call alongside the existing
  GetNativeCoinRow. Drift-safe — GetCoinByAssetID is the
  exact reader the /v1/assets/{id} handler calls
  (assets_coin_extension.go:215); same wire path = same cache
  key.