Skip to content

@subsquid/pipes v1.0.0-beta.4

Choose a tag to compare

@github-actions github-actions released this 20 Aug 12:36
· 9 commits to main since this release

Pub/Sub signal routes and commit-lag metrics

The Pub/Sub target can now publish plain application messages alongside CDC rows, and reports the same block-to-commit lag the BigQuery target does. One metric was renamed — see Breaking.

Breaking

  • sqd_pubsub_ops_total is renamed to sqd_pubsub_operations_total. The counter gained a kind label to distinguish CDC rows from signals, and a label set cannot be widened in place. Dashboards and alerts on the old name have to be repointed.

Targets

  • Pub/Sub signal routes — a signals route publishes the payload verbatim, with no _id / _CHANGE_TYPE / _CHANGE_SEQUENCE_NUMBER envelope. Use it when the consumer is not a table: a stateful fold, a control channel, a watermark announcement. A signal has no row identity, so a fork cannot be repaired per message; each route declares how it copes — boundary publishes one durable message per fork carrying an epoch the consumer reconciles against, finalized-only refuses to publish anything a fork could orphan (a draft above the finalized head is a fatal E2427). A stream can feed both a topics and a signals route. See docs/examples/evm/19.pubsub-signals.example.ts.
  • Pub/Sub commit-lag histogramssqd_pubsub_block_to_commit_lag_seconds and sqd_pubsub_portal_to_commit_lag_seconds. Both anchor on the Pub/Sub ack and are observed even when the publish throws, so an outage makes the series climb instead of falling silent.
  • BigQuery block-to-commit lag no longer trusts the raw block time. Portal block timestamps are per-network — tron and substrate report milliseconds — and a tron value above 2^53 used to freeze the histogram's _sum for the life of the process. Values are normalized to seconds, and an implausible one is skipped rather than observed.

Portal client

  • Cached batches hand back meta.lastBlockReceivedAt as a Date again. The JSON round-trip through the portal cache returned it as a string, so consumers calling .getTime() on a cache hit got a runtime error.

API

  • blockTimestampSeconds(timestamp) — normalizes a portal block timestamp to epoch seconds, or undefined when it cannot be trusted.
  • New Pub/Sub types: SignalRoute, SignalDraft, PendingCdcOperation, PendingSignalOperation, SignalForkContext.
  • New error codes: E2425 (no routes configured), E2426 (signal draft without chain attribution), E2427 (finalized-only signal from an unfinalized block). E2107 is now documented in docs/errors.md.

Full Changelog: pipes-v1.0.0-beta.3...pipes-v1.0.0-beta.4