@subsquid/pipes v1.0.0-beta.4
·
9 commits
to main
since this release
Pub/Sub signal routes and commit-lag metrics
The Pub/Sub target can now publish plain application messages alongside CDC rows, and reports the same block-to-commit lag the BigQuery target does. One metric was renamed — see Breaking.
Breaking
sqd_pubsub_ops_totalis renamed tosqd_pubsub_operations_total. The counter gained akindlabel to distinguish CDC rows from signals, and a label set cannot be widened in place. Dashboards and alerts on the old name have to be repointed.
Targets
- Pub/Sub signal routes — a
signalsroute publishes the payload verbatim, with no_id/_CHANGE_TYPE/_CHANGE_SEQUENCE_NUMBERenvelope. Use it when the consumer is not a table: a stateful fold, a control channel, a watermark announcement. A signal has no row identity, so a fork cannot be repaired per message; each route declares how it copes —boundarypublishes one durable message per fork carrying an epoch the consumer reconciles against,finalized-onlyrefuses to publish anything a fork could orphan (a draft above the finalized head is a fatalE2427). A stream can feed both atopicsand asignalsroute. Seedocs/examples/evm/19.pubsub-signals.example.ts. - Pub/Sub commit-lag histograms —
sqd_pubsub_block_to_commit_lag_secondsandsqd_pubsub_portal_to_commit_lag_seconds. Both anchor on the Pub/Sub ack and are observed even when the publish throws, so an outage makes the series climb instead of falling silent. - BigQuery block-to-commit lag no longer trusts the raw block time. Portal block timestamps are per-network — tron and substrate report milliseconds — and a tron value above 2^53 used to freeze the histogram's
_sumfor the life of the process. Values are normalized to seconds, and an implausible one is skipped rather than observed.
Portal client
- Cached batches hand back
meta.lastBlockReceivedAtas aDateagain. The JSON round-trip through the portal cache returned it as a string, so consumers calling.getTime()on a cache hit got a runtime error.
API
blockTimestampSeconds(timestamp)— normalizes a portal block timestamp to epoch seconds, orundefinedwhen it cannot be trusted.- New Pub/Sub types:
SignalRoute,SignalDraft,PendingCdcOperation,PendingSignalOperation,SignalForkContext. - New error codes:
E2425(no routes configured),E2426(signal draft without chain attribution),E2427(finalized-only signal from an unfinalized block).E2107is now documented indocs/errors.md.
Full Changelog: pipes-v1.0.0-beta.3...pipes-v1.0.0-beta.4