Repository navigation
Releases
v0.12.0
blobs sink moves to the official Substreams sink library and takes its block range from flags, which allows starting at the chain head. blobs api, blobs migrate-kv and the Substreams package (eth-blobs-v0.10.0.spkg) are unchanged. Image: ghcr.io/pinax-network/eth-blobs:v0.12.0.
Breaking: blobs sink command line
| v0.11.0 | v0.12.0 |
|---|---|
blobs sink <endpoint> <spkg> <start>: |
blobs sink <endpoint> <spkg> --start-block=<start> |
blobs sink <endpoint> <spkg> <start>:<stop> |
… --start-block=<start> --stop-block=<stop> (-s, -t; <stop> stays exclusive) |
--infinite-retry |
--max-retries=-1 |
--irreversible-only |
--final-blocks-only |
A leftover range argument fails at startup and names the new flags. Update the arguments in the same change that moves a deployment to this image.
New: start at the chain head
A negative --start-block is resolved by the server against the chain head: --start-block=-1 starts at the block below it. A new writer with an empty bucket can start live without anyone looking up a current slot. Starting at the head is also the quickest way to exercise the live path (non-final blocks, undo signals, missed slots) when testing locally.
As before, the start block applies only while the writer has no cursor/<writer> object; with one, the sink resumes from it. A head-relative start is not reproducible, so a writer that must line up with a backfill keeps an absolute start.
Changed: official sink library (#16)
The sink now uses github.com/streamingfast/substreams/sink instead of a fork of the deprecated github.com/streamingfast/substreams-sink. Behaviour is the same: there is still no undo buffer, metrics are still served on --metrics-listen-addr, and --max-retries defaults to 15 (about 5 minutes of backoff), the previous budget.
- New flags from the library:
--max-retries,--force-protocol-version,--proto-path,--proto-descriptor-set. - Metrics:
substreams_sink_data_message_size_bytesandsubstreams_sink_progress_message_total_processed_blocksare gone; the closest replacements aresubstreams_sink_server_egress_bytesandsubstreams_sink_processed_blocks. The library also addssubstreams_sink_processed_bytes,substreams_sink_block_wait_time,substreams_sink_block_time_deltaandsubstreams_sink_local_processing_time.head_block_number,head_block_time_drift, the othersubstreams_sink_*metrics and theblobs_sink_*metrics are unchanged.
Full changelog: v0.11.0...v0.12.0
v0.11.0
One Go binary, blobs, replaces api-service and the substreams-sink-kv deployment it read from. It writes the map_blobs output to an object store and serves the Beacon blob endpoints from there. Image: ghcr.io/pinax-network/eth-blobs:v0.11.0.
| Command | Does |
|---|---|
blobs sink <endpoint> <spkg> [<start>:<stop>] |
streams map_blobs into the store and handles every reorg itself |
blobs api --config config.yaml |
serves /eth/v1/beacon/blob_sidecars/{block_id}, /eth/v1/beacon/blobs/{block_id} and /health; same routes and responses as api-service |
blobs migrate-kv <kv-serve-addr> [<from>:<to>] |
copies history out of a running substreams-sink-kv serve |
The store is a dstore URL (--store-url, store.url): S3-compatible endpoints (Ceph RGW, MinIO), AWS S3, GCS, Azure (prefix required) or a local directory. Credentials come from each backend's standard environment. See blobs/README.md for the object layout and AGENTS.md for the invariants.
Breaking changes
api-service/is gone. The image's entrypoint is now/app/blobsand takes a subcommand (api,sinkormigrate-kv); the old/app/blob-serviceentrypoint no longer exists.- API config:
sink.addressis replaced bystore.url(plus optionalstore.timeoutandstore.head_ttl).application.grpc_hostis removed; that gRPC server only exposed health and reflection. - The Go module is
github.com/pinax-network/blob-service/blobs.
Migrating an existing deployment
- Run
blobs migrate-kvagainst the existingsubstreams-sink-kv serveto copy history into the store. It is rerunnable: slots already present are skipped and their root index checked, and every KV read and store operation is retried (--retries, default 8). - Start
blobs sinkwith a start block of the migrated range's end + 1, one writer id per sink. - Point
blobs apiat the same store.
Also in this release
HexBytesis encoded straight into the JSON output: about 18% less CPU and 2.5× fewer allocated bytes on fullblob_sidecarsresponses.- Slot objects are zstd-compressed (
slots/<slot>.pb.zst), about 20% smaller on mainnet. - Every sink store write (batch, head, cursor, undo deletes) is retried with backoff before the sink exits.
- For
s3://URLs the SDK computes and validates checksums only when required, which S3-compatible servers need.
The Substreams package is unchanged: eth-blobs-v0.10.0.spkg remains current.
v0.10.0
What's Changed
- Support Gloas blocks in blobs substreams by @YaroShkvorets in #12
Full Changelog: v0.9.3...v0.10.0
v0.9.3
v0.9.2
What's Changed
- Improve query array parsing and add tests by @YaroShkvorets in #11
Full Changelog: v0.9.1...v0.9.2
v0.9.1
What's Changed
- Add finality detection based on configurable slot lag by @YaroShkvorets in #10
Full Changelog: v0.9.0...v0.9.1
v0.9.0
What's Changed
- Add Beacon API v4.0.0 blobs endpoint with versioned hash filtering by @YaroShkvorets in #9
Full Changelog: v0.8.2...v0.9.0
v0.8.2
What's Changed
- Blob validator by @YaroShkvorets in #6
- Fix CORS by @YaroShkvorets in #7
Full Changelog: v0.8.1...v0.8.2
v0.8.1
Make HTTPS default scheme in swagger
Full Changelog: v0.8.0...v0.8.1
v0.8.0
What's Changed
- Update to latest substreams version by @YaroShkvorets in #5
Full Changelog: v0.7.1...v0.8.0