Skip to content

Releases

v0.12.0

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 09 Nov 23:05
784de5d

blobs sink moves to the official Substreams sink library and takes its block range from flags, which allows starting at the chain head. blobs api, blobs migrate-kv and the Substreams package (eth-blobs-v0.10.0.spkg) are unchanged. Image: ghcr.io/pinax-network/eth-blobs:v0.12.0.

Breaking: blobs sink command line

v0.11.0 v0.12.0
blobs sink <endpoint> <spkg> <start>: blobs sink <endpoint> <spkg> --start-block=<start>
blobs sink <endpoint> <spkg> <start>:<stop> … --start-block=<start> --stop-block=<stop> (-s, -t; <stop> stays exclusive)
--infinite-retry --max-retries=-1
--irreversible-only --final-blocks-only

A leftover range argument fails at startup and names the new flags. Update the arguments in the same change that moves a deployment to this image.

New: start at the chain head

A negative --start-block is resolved by the server against the chain head: --start-block=-1 starts at the block below it. A new writer with an empty bucket can start live without anyone looking up a current slot. Starting at the head is also the quickest way to exercise the live path (non-final blocks, undo signals, missed slots) when testing locally.

As before, the start block applies only while the writer has no cursor/<writer> object; with one, the sink resumes from it. A head-relative start is not reproducible, so a writer that must line up with a backfill keeps an absolute start.

Changed: official sink library (#16)

The sink now uses github.com/streamingfast/substreams/sink instead of a fork of the deprecated github.com/streamingfast/substreams-sink. Behaviour is the same: there is still no undo buffer, metrics are still served on --metrics-listen-addr, and --max-retries defaults to 15 (about 5 minutes of backoff), the previous budget.

  • New flags from the library: --max-retries, --force-protocol-version, --proto-path, --proto-descriptor-set.
  • Metrics: substreams_sink_data_message_size_bytes and substreams_sink_progress_message_total_processed_blocks are gone; the closest replacements are substreams_sink_server_egress_bytes and substreams_sink_processed_blocks. The library also adds substreams_sink_processed_bytes, substreams_sink_block_wait_time, substreams_sink_block_time_delta and substreams_sink_local_processing_time. head_block_number, head_block_time_drift, the other substreams_sink_* metrics and the blobs_sink_* metrics are unchanged.

Full changelog: v0.11.0...v0.12.0

v0.11.0

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 09 Nov 19:00
d619c94

One Go binary, blobs, replaces api-service and the substreams-sink-kv deployment it read from. It writes the map_blobs output to an object store and serves the Beacon blob endpoints from there. Image: ghcr.io/pinax-network/eth-blobs:v0.11.0.

Command Does
blobs sink <endpoint> <spkg> [<start>:<stop>] streams map_blobs into the store and handles every reorg itself
blobs api --config config.yaml serves /eth/v1/beacon/blob_sidecars/{block_id}, /eth/v1/beacon/blobs/{block_id} and /health; same routes and responses as api-service
blobs migrate-kv <kv-serve-addr> [<from>:<to>] copies history out of a running substreams-sink-kv serve

The store is a dstore URL (--store-url, store.url): S3-compatible endpoints (Ceph RGW, MinIO), AWS S3, GCS, Azure (prefix required) or a local directory. Credentials come from each backend's standard environment. See blobs/README.md for the object layout and AGENTS.md for the invariants.

Breaking changes

  • api-service/ is gone. The image's entrypoint is now /app/blobs and takes a subcommand (api, sink or migrate-kv); the old /app/blob-service entrypoint no longer exists.
  • API config: sink.address is replaced by store.url (plus optional store.timeout and store.head_ttl). application.grpc_host is removed; that gRPC server only exposed health and reflection.
  • The Go module is github.com/pinax-network/blob-service/blobs.

Migrating an existing deployment

  1. Run blobs migrate-kv against the existing substreams-sink-kv serve to copy history into the store. It is rerunnable: slots already present are skipped and their root index checked, and every KV read and store operation is retried (--retries, default 8).
  2. Start blobs sink with a start block of the migrated range's end + 1, one writer id per sink.
  3. Point blobs api at the same store.

Also in this release

  • HexBytes is encoded straight into the JSON output: about 18% less CPU and 2.5× fewer allocated bytes on full blob_sidecars responses.
  • Slot objects are zstd-compressed (slots/<slot>.pb.zst), about 20% smaller on mainnet.
  • Every sink store write (batch, head, cursor, undo deletes) is retried with backoff before the sink exits.
  • For s3:// URLs the SDK computes and validates checksums only when required, which S3-compatible servers need.

The Substreams package is unchanged: eth-blobs-v0.10.0.spkg remains current.

v0.10.0

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 06 Oct 16:03
d4a5c9c

What's Changed

Full Changelog: v0.9.3...v0.10.0

v0.9.3

Choose a tag to compare

@gcleroux gcleroux released this 28 Aug 20:11

Full Changelog: v0.9.2...v0.9.3

v0.9.2

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 20 May 16:01
7650d8e

What's Changed

Full Changelog: v0.9.1...v0.9.2

v0.9.1

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 13 May 21:40
99cd60b

What's Changed

Full Changelog: v0.9.0...v0.9.1

v0.9.0

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 13 May 19:34
46943e6

What's Changed

  • Add Beacon API v4.0.0 blobs endpoint with versioned hash filtering by @YaroShkvorets in #9

Full Changelog: v0.8.2...v0.9.0

v0.8.2

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 30 Apr 21:17
be836df

What's Changed

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 20 Oct 16:36
9d29930

Make HTTPS default scheme in swagger

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Choose a tag to compare

@YaroShkvorets YaroShkvorets released this 20 Oct 15:53
bd91531

What's Changed

Full Changelog: v0.7.1...v0.8.0