Releases: subsquid-labs/portal-ponder
Release list
@subsquid/ponder 0.17.6-sqd.1
@subsquid/ponder@0.17.6-sqd.1 — the Portal fork of ponder, grafted onto ponder@0.17.6.
What this is
The Portal layer (backfill from SQD Portal) grafted onto upstream ponder@0.17.6, published as a drop-in ponder bin. Routine upstream-parity: tracks a new upstream patch line at parity, no fork behavior or public-surface change.
Upstream delta 0.17.4 → 0.17.6
- The graft seam (
HistoricalSync.syncBlockRangeData/syncBlockData, interval-scoped) is unchanged. - The only wiring re-derivation is
rpc/actions.ts: upstream moved address-chunking intoeth_getLogs, so the Portal helper block re-anchors after the now-chunkedeth_getLogs(11/12 hunks verbatim, 1 re-anchored).eth_getFilteredBlockLogsdepends only oneth_getLogs's unchanged in/out contract, so the chunking is a transparent internal detail. - Upstream's null-topic / zeroHash / address-chunking changes are confined to the RPC-mode plane, off the Portal
/streampath.
Gate
Both-version scripts/sync-upstream.sh --test green (0.17.6 graft + supported lines); biome clean; no exported-type / config / signature drift vs 0.17.4-sqd.1. CI fork-seam(0.17.6) green on the tracking PR (#216).
Published with --tag latest (numeric-max) — the default npm install @subsquid/ponder. 0.17.4-sqd.1 remains installable by exact version.
@subsquid/ponder 0.17.4-sqd.1
@subsquid/ponder@0.17.4-sqd.1
ponder@0.17.4 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is an upstream-parity release: 0.17.3 → 0.17.4 is four upstream indexing-performance patch PRs, every one off-seam and grep-proven inert for the fork, so they are passively inherited with no Portal-layer behavior change and no wiring change. It installs by default (--tag latest), superseding 0.17.3-sqd.1 (which now has no dist-tag — installable by exact version, npm i @subsquid/ponder@0.17.3-sqd.1).
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.17.4-sqd.1.
What changed upstream — ponder@0.17.3 → 0.17.4
Four upstream patch-level indexing-performance PRs (all @kyscott18), passively inherited:
ponder-sh/ponder#2353(a0a91c3) — batch internal metrics updates.#2356(13d34ee) — schema-aware copying.#2355(d34cf7e) — normalize only changed row fields on updates.#2352(0183ecc) — reduce copy-on-write row memory.
The entire packages/core/src delta is nine files, all in ponder's indexing plane: indexing-store/{cache,index,utils}.ts, internal/metrics.ts, utils/copy.ts, indexing/index.ts (two call-sites swapped to a batched metrics helper), and a one-line common.metrics.flushIndexingMetrics() added at the tail of each runtime/{isolated,multichain,omnichain}.ts indexing loop.
Three of the nine changed files (runtime/isolated|multichain|omnichain.ts) are among the 12 wiring files, but each has a single wiring hunk at its run* entrypoint (L152 / L174 / L177) — above and non-overlapping the perf additions (L401 / L454 / L473). So portal/wiring/0.17.4.patch — a verbatim, byte-identical copy of portal/wiring/0.17.3.patch (same 12 files; the verbatim lineage now runs 0.17.0 → 0.17.1 → 0.17.2 → 0.17.3 → 0.17.4) — applies with zero rejects. The six load-bearing wiring files (build/config, config/index, internal/types, rpc/actions, rpc/index, runtime/historical — the historical seam) are byte-identical 0.17.3 → 0.17.4.
Grep-proven inert for the fork. The Portal layer (portal/*.ts) references none of the three changed metrics symbols (ponder_indexing_completed_events / incrementIndexingCompletedEvents / flushIndexingMetrics), and the upstream perf changes are confined to the indexing-store / metrics / copy internals that the Portal fetch/assembly layer never touches. Dependencies are unchanged except the @ponder/client self-version bump. The graft seam (HistoricalSync.syncBlockRangeData / syncBlockData, interval-scoped) is unchanged; 0.17.4 touches only the indexing plane.
Compatibility
- Built + tested against
ponder@0.17.4:scripts/sync-upstream.sh 0.17.4 --test— patch applies clean (zero rejects),tscbuilds, 416 tests / 24 files green; both-version confirmed (the identical suite is also green onponder@0.16.6, the pnpm-9 path). CI runs thefork-seam(0.17.4)job (incompat.tested). - No fresh RPC byte-diff is claimed on
0.17.4— compatibility rests on seam identity + verbatim patch + the full suite, exactly as for0.17.3-sqd.1. The§3 / §5byte-diff and A/B soak evidence remains on the0.16.6graft and transfers by seam identity. - Seam verified identical across
0.15.17–0.17.4(compat.testedinversions.json).
@subsquid/ponder 0.17.3-sqd.1
@subsquid/ponder@0.17.3-sqd.1
ponder@0.17.3 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is an upstream-parity release: 0.17.2 → 0.17.3 is a single upstream memory-retention bugfix, off-seam and grep-proven inert for the fork, so it is passively inherited with no Portal-layer change and no wiring change. It installs by default (--tag latest), superseding 0.17.2-sqd.1 (which now has no dist-tag — installable by exact version, npm i @subsquid/ponder@0.17.2-sqd.1).
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.17.3-sqd.1.
What changed upstream — ponder@0.17.2 → 0.17.3
One upstream patch-level bugfix, passively inherited:
ponder-sh/ponder#2350(@kyscott18, commit80dd936) — fixes realtime sync retaining unfiltered block transactions in memory until finalization (upstream reports >10% live-indexing memory improvement). The entire source delta is one line insrc/sync-realtime/index.ts'sreconcileBlockasync generator (~L1140): the yielded block'stransactionsarray is repointed from the unfiltered set to the already-filtered top-leveltransactionsarray, so the unfiltered array is released for garbage collection.
src/sync-realtime/index.ts is one of the 12 wiring files, but the #2350 edit (~L1140) does not overlap the wiring patch's two hunks (the import block and the #23 filtered-logs shouldRequestLogs fallback, ~L308–347), so portal/wiring/0.17.3.patch — a verbatim copy of portal/wiring/0.17.2.patch (sha256-equal, same 12 files; the verbatim lineage now runs 0.17.0 → 0.17.1 → 0.17.2 → 0.17.3) — applies with zero rejects.
Grep-proven inert for the fork. The swap changes what a yielded realtime event's block.transactions contains (unfiltered → filtered), but nothing in the fork reads it: .block.transactions is referenced zero times in the Portal layer (portal/*.ts) and in the wiring patch. Realtime transactions are persisted from the top-level transactions field (filtered, identical logic in 0.17.2 and 0.17.3), and the #27 access_list path maps from the top-level Portal transactions, never from block.transactions. So the change persists and emits no different data on either the RPC or Portal /stream path — it only frees memory. The graft seam (HistoricalSync.syncBlockRangeData / syncBlockData, interval-scoped) is unchanged; 0.17.3 touches only src/sync-realtime/index.ts.
Compatibility
- Built + tested against
ponder@0.17.3:scripts/sync-upstream.sh 0.17.3 --test— patch applies clean (all 12 files "Applied … cleanly"),tscbuilds, 408 tests / 23 files green; both-version confirmed (the identical suite is also green onponder@0.17.2). CI runs thefork-seam(0.17.3)job (incompat.tested). - No fresh RPC byte-diff is claimed on
0.17.3— compatibility rests on seam identity + verbatim patch + the full suite, exactly as for0.17.2-sqd.1. The§3 / §5byte-diff and A/B soak evidence remains on the0.16.6graft and transfers by seam identity. - Seam verified identical across
0.15.17–0.17.3(compat.testedinversions.json).
@subsquid/ponder 0.17.2-sqd.1
@subsquid/ponder@0.17.2-sqd.1
ponder@0.17.2 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is an upstream-parity release: 0.17.1 → 0.17.2 is two upstream patch-level bugfixes, both off-seam or non-overlapping with the graft, so the fork passively inherits them with no Portal-layer change and no wiring change. It installs by default (--tag latest), superseding 0.17.1-sqd.2 (which now has no dist-tag — installable by exact version, npm i @subsquid/ponder@0.17.1-sqd.2).
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.17.2-sqd.1.
What changed upstream — ponder@0.17.1 → 0.17.2
Two upstream patch-level bugfixes, both passively inherited:
ponder-sh/ponder#2341(@hellowodl) — fixes rows returned bycontext.db.find()/insert()/update()being mutated by a subsequent write to the same row (adds a defensive copy at the get-sites). Implemented purely insrc/indexing-store/index.ts, which is off the graft surface (the indexing store, like the0.16.9bytea-COPY fix) — the seam is untouched.ponder-sh/ponder#2347(@kyscott18) — fixes realtime-discovered factory child addresses being omitted fromponder_sync.factory_addresseswhen a factory is shared across multiple event handlers. Implemented insrc/sync-realtime/index.ts— one of the 12 wiring files, but in regions that do not overlap the wiring patch's two hunks (the import block and the#23filtered-logsshouldRequestLogsfallback), soportal/wiring/0.17.2.patchapplies with zero rejects and the fork passively inherits the upstream completeness fix. This is a realtime runtime-state delta (a strictly larger persistedfactory_addressesset for shared factories), inherited verbatim from upstream — not a behavior change introduced by the Portal layer.
portal/wiring/0.17.2.patch is a verbatim copy of portal/wiring/0.17.1.patch (sha256-equal, same 12 files — itself a verbatim copy back to 0.17.0). The graft seam (HistoricalSync.syncBlockRangeData / syncBlockData, interval-scoped) is unchanged; 0.17.2 touches only src/indexing-store/index.ts and src/sync-realtime/index.ts, neither of which is the historical seam.
Compatibility
- Built + tested against
ponder@0.17.2:scripts/sync-upstream.sh 0.17.2 --test— patch applies clean (all 12 files "Applied … cleanly"),tscbuilds, 396 tests / 22 files green; both-version confirmed (the identical suite is also green onponder@0.16.6). - No fresh RPC byte-diff is claimed on
0.17.2— compatibility rests on seam identity + verbatim patch + the full suite, exactly as for0.17.1-sqd.2. The§3 / §5byte-diff and A/B soak evidence remains on the0.16.6graft and transfers by seam identity. - Seam verified identical across
0.15.17–0.17.2(compat.testedinversions.json).
@subsquid/ponder 0.17.1-sqd.2
@subsquid/ponder@0.17.1-sqd.2
ponder@0.17.1 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is a fork-side fix release on the same ponder@0.17.1 — the wiring patch and upstream version are unchanged from 0.17.1-sqd.1; only the Portal layer gained the fix below. It installs by default (--tag latest), superseding 0.17.1-sqd.1 (which now has no dist-tag — installable by exact version, npm i @subsquid/ponder@0.17.1-sqd.1).
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.17.1-sqd.2.
What changed since 0.17.1-sqd.1 — factory query-sharding (#194, PR #195)
A factory whose discovered child-address set is large (~5.8k+) overflowed Portal's 256 KiB raw-query cap (MAX_RAW_QUERY_SIZE) in a single query body → Portal 400 → the fork's fail-loud turned that into a permanent hard stop for the chain. Batching + merge don't reduce the total body size, so above the wall the chain could not progress.
The fix — byte-budgeted log-query sharding. logQueryShards() partitions the merged log-request address union into multiple query bodies, each kept below the cap, streamed sequentially and unioned:
- Completeness preserved by construction (the Portal layer's INV-1 / INV-3 / INV-11 filter-completeness invariants): the shard loop lives inside the chunk's
runStreams, so the chunk is cached/marked-done only after every shard has drained — no partial-commit window — and any shard error rejects the whole chunk and retries the full plan. - Byte-identical no-op below the wall: when the whole body fits one shard the result is exactly one shard equal to the un-sharded query, so the entire existing validated corpus (≤ 872 children) is provably unaffected.
- A single merged element that alone exceeds the budget is now an attributable, up-front error rather than an opaque 400 at stream time.
Mutation-verified: a mutation that drops all-but-the-first shard (the silent-loss failure mode) turns the cross-shard completeness tests red.
Scope: historical-plane log/factory-child sharding. Realtime /stream sharding, factory→tx-filter sharding, and shard concurrency are tracked follow-ups (#196) — each retains a safe fail-loud (or the RPC-realtime fallback) today.
Compatibility
- Built + tested against
ponder@0.17.1:scripts/sync-upstream.sh 0.17.1 --test— patch applies clean,tscbuilds, 379 tests / 20 files green (addsportal/portal-shard.test.ts). - Wiring patch and graft seam (
HistoricalSync.syncBlockRangeData/syncBlockData) unchanged from0.17.1-sqd.1; the fix is off-seam Portal-layer code (portal/portal-filters.ts,portal/portal.ts). - Seam verified identical across
0.15.17–0.17.1(compat.testedinversions.json).
Previous revision — 0.17.1-sqd.1 (upstream-parity bump)
No Portal-layer change — 0.17.1-sqd.1 was a pure upstream-parity bump. 0.17.0 → 0.17.1 is a single upstream performance PR (ponder-sh/ponder#2338, @kyscott18: "Improved sync performance by removing redundant block range constraints from event queries"), which narrows Ponder's internal trace/log read-back query (sync-store/index.ts, getEventData) to the block ranges of the filters active in each interval. That change sits downstream of the Portal fetch/insert and off the graft seam, so the rows the Portal writes are unchanged. portal/wiring/0.17.1.patch is a verbatim copy of portal/wiring/0.17.0.patch (sha256-equal, same 12 files).
@subsquid/ponder 0.17.0-sqd.1
@subsquid/ponder@0.17.0-sqd.1
ponder@0.17.0 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is the
first cut on ponder@0.17.0 and the newest ponder version in the fork, so it installs by
default. It carries the same Portal layer as 0.16.10-sqd.1 — the full invariant-first
re-architecture and correctness wave — grafted onto ponder@0.17.0, plus the fork-side fixes
below that landed since the last release note (0.16.8-sqd.1). It supersedes 0.16.9-sqd.1
as latest (that cut had no release note; it tracked ponder@0.16.9's upstream bytea-COPY
fix — see Compatibility).
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.17.0-sqd.1.
Highlights (new since 0.16.8-sqd.1)
- Realtime finalize is write-side idempotent on resume — no duplicated factory-child rows
(INV-17, #175). A resumed single writer that re-finalizes the same factory-child creation
block used to insert theponder_sync.factory_addressesrow twice (once by the pre-kill
finalize transaction, once by the resume finalize — two sequential finalize transactions).
The read→dedupe guard (read the store's persisted children, keep a child only if it is absent
or re-discovered at a strictly lower block — LEAST semantics) previously ran only at the
historical call site; it now also runs on the realtime finalize path, so both sync modes are
byte-identically idempotent on resume. App data was already unaffected (the only consumer
min-merges to a set) — this closes the store-identity/table-growth defect under repeated
realtime crash/resume. inserted.logsreports the store-inserted count, not the raw-streamed count (#143, #177).
Theinserted.logsmetric (and the completion line'slogs=) now counts the assembled logs
actually written to the store, instead of the raw pre-re-match streamed count. A window whose
over-returned logs are all re-match-dropped now reportsinserted.logs=0, consistent with
inserted.blocks=0. This is a metric-accuracy fix — no change to the data that is stored.
Portal layer
- Write-side idempotence (
portal-child-dedupe.ts,runtime/realtime.tswiring hook): the
shared read→dedupe core (dedupeChildAddressesAgainstStore) now runs at BOTH factory-child
insert call sites — the historical path (portal.tspersistPendingChildren) and the
realtime finalize path (runtime/realtime.tshandleRealtimeSyncEvent, via
dedupeFinalizeChildAddresses), each inside its own store transaction (INV-17, #175). - Metrics (
portal-metrics.ts,portal.ts):inserted.logsis mapped to a new
insertedLogscounter taken fromassembled.logs.lengthat theinsertLogssite; the raw
logscount is retained for the progress fingerprint andmb_streamedkeeps raw-I/O
semantics (#143, #177). - Wiring patch:
portal/wiring/0.17.0.patch. It is re-derived, not verbatim:0.16.10 → 0.17.0is a tree-wide mechanical reformat (import-list reorder,static readonly→
static override readonly,@ts-ignore→@ts-expect-error) plus dep bumps (@ponder/utils
0.2.18 → 0.3.0,pg^8.11 → ^8.16). Applying the0.16.10patch left 29 of 34 hunks landing
by offset and 5 rejected — all 5 pure import-block additions where the reformat reordered the
surrounding import lists; each rejected hunk's added lines were re-placed byte-for-byte into
the reformatted import block (comm-verified: the set of added wiring lines is identical to
0.16.10.patch, zero delta). The graft seam
(HistoricalSync.syncBlockRangeData/syncBlockData, interval-scoped) survives untouched.
Fixes
Everything in 0.16.8-sqd.1 (the full phantom-coverage / stream-realtime+reorg / factory-and-
account-source / bounded-warmup / HTTP-client correctness wave, plus the accessList
honest-NULL fix and the default-level Portal logging), plus the two items above: INV-17
realtime-finalize write-side idempotence (#175) and the inserted.logs store-inserted metric
(#143, #177). The fork also passively inherits two off-seam upstream fixes over this range:
ponder@0.16.9's bytea-COPY encoding fix (ponder-sh/ponder#2323, in indexing-store/cache.ts)
and ponder@0.16.10's commit-path cache-invalidation resilience (runtime/isolated.ts /
multichain.ts / omnichain.ts) — both off the graft surface, inherited unchanged.
Compatibility
- Built + tested against
ponder@0.17.0. The full Portal property/regression suite is green:
370/370 across 19 files onponder@0.17.0(scripts/sync-upstream.sh 0.17.0 --test) —
and identically on every tracked version (0.15.17,0.16.6,0.16.7,0.16.8,0.16.9,
0.16.10,0.17.0). ponder@0.17.0bumps its build toolchain (engines.pnpm>=11,packageManagerpnpm@11.0.0)
and its deps (@ponder/utils0.3.0,pg^8.16); the fork inherits those upstream deps with no
fork-side dep change. Thesync-upstream.shharness reads each clone's ownpackageManagerand
builds with the pnpm it declares (falling back to the historical pnpm@9.10.0 pin when the field
is absent), so0.16.10still builds with pnpm 9.10.0 and0.17.0with pnpm 11 — both green at
370 tests.- No fresh RPC byte-diff on
0.17.0; compat rests on seam-identity (the wiring patch re-derives
but re-places every added wiring line byte-for-byte, so the graft is unchanged) plus the full
suite, as for0.16.10/0.16.9/0.15.17. - Seam verified against:
0.17.0,0.16.10,0.16.9,0.16.8,0.16.7,0.16.6,0.15.17
(compat.tested). The seam is stable and identical in shape across0.15.17 – 0.17.0.
@subsquid/ponder 0.16.10-sqd.2
@subsquid/ponder@0.16.10-sqd.2
Supported-lines backport wave (2026-07-28). -sqd.2 re-cuts ponder@0.16.10 with the current Portal layer — the same portal/ tree shipped on latest (0.17.2-sqd.1). It keeps 0.16.10 a first-class supported line so a user pinned to ponder-0.16.10 runs the same fixes as everyone else.
Published under dist-tag ponder-0.16.10 — latest is not clobbered (it stays 0.17.2-sqd.1, the newest tracked ponder version). Install: npm i @subsquid/ponder@0.16.10-sqd.2 (or npm i @subsquid/ponder@ponder-0.16.10).
New since 0.16.10-sqd.1
- Factory log-query byte-budget sharding (#194, #195). A factory whose discovered child-address IN-list union overflows the Portal 256 KiB query cap now splits the over-cap child set into byte-sized shards, streamed sequentially and unioned — completeness-preserving by construction, a byte-identical no-op below the wall.
- Account-tx (from/to) byte-budget sharding (#196, #199). The same over-cap sharding applied to account-source transaction (
from/to) queries. - INV-26 store-state partition-invariance. The assembled store state is codified as invariant to how the Portal range- and shard-partitions a request (cross-shard duplicate logs collapse via
logs_pkey+onConflictDoNothing); adversarial-partition fuzz coverage added. Invariant catalog is now INV-1…INV-26.
Everything from 0.16.10-sqd.1 is carried forward (INV-17 realtime-finalize write-side idempotence #175, the inserted.logs store-inserted metric #143, the accessList honest-NULL fix, the full invariant-first correctness wave).
Compatibility
- Built + tested green on
ponder@0.16.10: 400/400 across 23 files (CI=true scripts/sync-upstream.sh 0.16.10 --test, pnpm@9.10.0) — verified both by a pre-publish gate and by the release workflow's own in-CI build+test. - The graft seam (
HistoricalSync.syncBlockRangeData/syncBlockData) is untouched; every added fix lives in the Portal fetch/assembly layer, off the seam.wiring/0.16.10.patchis unchanged (a verbatim copy ofwiring/0.16.9.patch). No fresh RPC byte-diff; compat rests on seam-identity + the full suite.
@subsquid/ponder 0.16.8-sqd.1
@subsquid/ponder@0.16.8-sqd.1
ponder@0.16.8 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is the
first cut on ponder@0.16.8 and the newest ponder version in the fork, so it installs by
default. It carries the same Portal layer as 0.16.7-sqd.1 — the full invariant-first
re-architecture and correctness wave — grafted onto ponder@0.16.8, plus the fixes below that
landed since the 0.16.7-sqd.1 cut.
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.16.8-sqd.1.
Highlights (new since 0.16.7-sqd.1)
accessListcorrectness — store NULL, never a fabricated[]. Some Portal datasets
(e.g. arbitrum, avalanche) don't carry theaccess_listcolumn. The transform previously
filled an empty[]for those rows; it now stores SQLNULL, byte-matching what an RPC node
reports for a transaction whose access list is genuinely absent — and the field is gated to the
exact EIP typed-transaction set{1,2,3,4}rather than "type ≥ 1", so a legacy tx is never
handed a spurious access list (#110, #111).- Positive Portal confirmation at default log level (closes #115). A backfill that routes
through the Portal now emits an INFO banner —Portal backfill active for <chain>: <url>— a
periodic progress ticker (blocks scanned / discovery chunks), and a completion line, and the
upstream "fetching backfill data" line is reworded to name SQD Portal on the Portal path. You
can now confirm from default logs that the Portal — not an RPC — served the history (#119). - Completion line reports Portal data-plane provenance (#131). The per-chain completion line
now carries the data-plane totals —logs/txs/receiptscounts and anrpc_fallbackcount —
and closes withserved entirely by the SQD Portal (0 JSON-RPC for history)exactly when
rpc_fallbackis0(otherwiseN block range(s) fell back to JSON-RPC). The claim is scoped to
the historical sync data plane; user-landreadContractstill uses your configured RPC. - Actionable persistent-throttle error instead of an
unhandledRejectioncrash (for #116).
A full backfill throttled (HTTP 429) by the free public Portal past its retry budget now
surfaces a clear, actionable error and holds a prefetch invariant lock, rather than exhausting
retries and crashing via an unhandled promise rejection (#118).
Portal layer
- Transform (
portal-transform.ts): theaccessListhonest-NULL fix (#110/#111) — see the
Highlights above. - Observability (
portal.ts,portal-metrics.ts): the default-level Portal banner, progress
ticker, and completion line (#119); the completion line reports data-plane provenance —
logs/txs/receiptscounts and anrpc_fallbackcount, with a0 JSON-RPC for history
clause gated onrpcFallback === 0(#131). - HTTP client (
portal-client.ts): persistent-throttle actionable error + prefetch lock (#118). - Wiring patch:
portal/wiring/0.16.8.patch. Byte-identical to the0.16.6/0.16.7
patches (sha256ddd12dd5…, same 10/10 files); it applies cleanly (zero rejects), builds, and
passes the full Portal suite onponder@0.16.8.0.16.7 → 0.16.8is a realtime-only upstream
change — it addsisAsyncExecutionChain(chainId 143/10143) insync-realtime/index.ts+
utils/finality.tsso async-execution chains can surface blocks beforelogsBloomis
execution-ready. Neither file is on the graft surface (sync-historical,runtime/historical,
sync-store,rpc,sync), so the seam is untouched and the 81 Portal realtime tests pass
unchanged.
Fixes
Everything in 0.16.7-sqd.1 (the full phantom-coverage / stream-realtime+reorg / factory-and-
account-source / bounded-warmup / HTTP-client correctness wave), plus the three items above:
accessList honest-NULL (#110, #111), default-level Portal logging (#119, closes #115), the
persistent-throttle actionable error + prefetch lock (#118, for #116), and the completion line's
Portal data-plane provenance (#131).
Compatibility
- Built + tested against
ponder@0.16.8. The full Portal property/regression suite is green:
312/312 acrossponder@0.15.17,ponder@0.16.6,ponder@0.16.7, andponder@0.16.8
(scripts/sync-upstream.sh <ver> --test) — grown from 302 at the0.16.7-sqd.1cut as the
correctness wave added coverage (incl. the #131 completion-line provenance tests). - No fresh RPC byte-diff on
0.16.8; compat rests on seam-identity (the wiring patch is
byte-identical to0.16.6/0.16.7) plus the full suite, as for0.16.7/0.15.17. - Seam verified against:
0.16.8,0.16.7,0.16.6,0.15.17(compat.tested). The seam is
stable and identical in shape across0.15.17 – 0.16.8.
@subsquid/ponder 0.16.7-sqd.1
@subsquid/ponder@0.16.7-sqd.1
ponder@0.16.7 + the SQD Portal backfill layer, drop-in (bin stays ponder). This is the
first cut on ponder@0.16.7 and the newest ponder version in the fork, so it installs by
default. It carries the same Portal layer as 0.16.6-sqd.2 — the full invariant-first
re-architecture and correctness wave — grafted onto ponder@0.16.7.
Install: npm i @subsquid/ponder, or pin exact: npm i @subsquid/ponder@0.16.7-sqd.1.
Highlights
- Silent-gap / phantom-coverage elimination. The known paths that could mark a block range
"synced" without actually delivering its data now fail loud (or retry) instead of recording
phantom coverage. This covers the
unbounded-source backfill floor (a source with nofromBlockstarts at genesis, not at the
min of the defined floors — INV-16), head-truncated frontier chunks (extended before being
served, never served stale — INV-13), a bounded backfill whosetoBlockruns past the
finalized head (head-clamped, not phantom-covered — INV-9/INV-13), and the mid-range
/finalized-stream204: a load-balanced replica serving only part of a requested range and
204-ing the tail is now a transient incomplete-range that retries on a fresher replica or
fails loud, never recorded as clean completion (issue #47). - Bounded time-to-first-durable-commit. The finalized-range fetch starts from a small warmup
span and grows geometrically, and on a restart resumes from the persisted watermark, so a dense
or wide-range source lands its first durable commit promptly instead of blocking on a full-range
scan — regardless of range size; discovery follows the same geometric slow-start on the
watermark-following path (#50). - Stream-realtime correctness + reorg handling. The unfinalized window is kept
strictly-increasing and parent-hash-linked; an unknown-parent gap is fatal (never a silent
fork switch), finality is verified by hash at the exact probe height, and a 1-block tip
orphan heals via the Portal's HTTP 409 fork-negotiation protocol instead of fataling
(issue #33). Same-block factory-child log redelivery is reconciled without dropping or
duplicating blocks (INV-10). - Factory / account-source coverage. Discovered factory children are persisted to the sync
store in the same transaction that marks the interval synced — including on the RPC
finality-delegation path — so a restart never silently drops a factory's events
(INV-15), with write-side idempotence so a re-flush never durably duplicates rows (INV-17).
Every transaction matched by a tx filter gets a receipt inserted (account sources). - Invariant-first architecture.
portal/is a pure functional core behind an imperative
shell; 18 named invariants (INV-1…INV-18,portal/INVARIANTS.md) are asserted at runtime and
proven by property-based + regression tests. A violation is a loudInvariantViolation, not
silent corruption.
Portal layer
- Backfill / transform (
portal.ts,portal-transform.ts,portal-client.ts,
portal-chunks.ts,portal-discovery.ts,portal-filters.ts,portal-assemble.ts): the
chunked finalized-range fetch, discovery-before-data ordering (INV-3), interval-exact
assembly (INV-2), full-tree trace ranking (INV-5), request merging (INV-11), AIMD concurrency
control + bounded buffered rows (INV-7/INV-8), and byte-identical RPC-shape transforms
(INV-6). - Realtime wiring (
portal-realtime.ts,portal-realtime-wire.ts): finalized-stream resume
withparentBlockHash, 409 fork negotiation, reorg reconciliation, and the
historical→realtime cutover guard (portal-cutover-guard.ts) that skips the refetch for
end-capped chains (INV-18). - Wiring patch:
portal/wiring/0.16.7.patch. Theruntime/historical.tscutover hunks are
byte-identical to the0.16.6patch; the patch applies cleanly (zero rejects) because the
grafted seam —HistoricalSync.syncBlockRangeData/syncBlockData, interval-scoped — is
identical in shape from 0.15.17 through 0.16.7.0.16.6 → 0.16.7is a single upstream
DB-layer change (live-query notification batching, ponder-sh/ponder#2314) that touches none
of the graft surfaces.
Fixes
The Portal-layer correctness fixes since the first fork cuts, grouped by theme:
- Silent-gap / phantom-coverage: genesis backfill floor for unbounded sources (#8, #10),
head-truncated frontier extension (#5), zero-line-200 gap (#66), bounded-backfill head clamp
and stream-mode delegation fatals (#44), and the mid-range 204 transient-incomplete-range
close (#47). - Stream-realtime + reorg: same-block child logs, finality anchor, reorg prune, head pin,
parent txs (#26);parentBlockHashon/streamresume + 1-block-orphan 409 healing (#33);
stream-mode finality floor at persisted finality (restart regression); omnichain
foreign-checkpoint timestamp mapping (#57); and the bounded historical→realtime cutover skip
(INV-18). - Factory / account-source coverage: persist discovered factory children (#2), flush on the
RPC-delegation path + survive core's transaction retry (INV-15), write-side idempotence
(#53), the factory-range gate pin (#21), and a receipt for every tx-filter-matched
transaction (#24); the extend-path needed-field fatal scoped to a genuine per-filter need — no
false-fatal on client-unmatched traces (#20) or a shared filter's receipt requirement (#94). - Liveness / bounded warmup: bounded time-to-first-durable-commit — the finalized-range
fetch (and the watermark-following discovery scan) start from a small warmup span and grow
geometrically, resume-seeded from the persisted watermark on restart (#50). - HTTP-client hardening: retry truncated response bodies + loud dataset-start skips (#25),
fail loud when a provider omitsaccessListon typed transactions (#27), and zero-backoff /
timeout hardening (#9 follow-ups, #16). - Invariant-first re-architecture: functional core + imperative shell (#6), with the
head-clamp / discovery-floor / trace-receipt correctness wave (#15) folded in.
Compatibility
- Built + tested against
ponder@0.16.7. The full Portal property/regression suite is green:
302/302 acrossponder@0.15.17,ponder@0.16.6, andponder@0.16.7
(scripts/sync-upstream.sh <ver> --test). - The dense-source / factory-child discovery paths are additionally re-accepted under a Poisson
kill/resume chaos campaign: the store stays
logically byte-identical to a clean baseline across 33 verified resumes with zero invariant
violations (VALIDATION.md§4.4). - Seam verified against:
0.16.7,0.16.6,0.15.17(compat.tested). The seam is stable
and identical in shape across0.15.17 – 0.16.7.
@subsquid/ponder 0.16.6-sqd.3
@subsquid/ponder@0.16.6-sqd.3
Supported-lines backport wave (2026-07-28). -sqd.3 re-cuts ponder@0.16.6 with the current Portal layer — the same portal/ tree shipped on latest (0.17.2-sqd.1). It keeps 0.16.6 a first-class supported line so a user pinned to ponder-0.16.6 runs the same fixes as everyone else.
Published under dist-tag ponder-0.16.6 — latest is not clobbered. Install: npm i @subsquid/ponder@0.16.6-sqd.3 (or npm i @subsquid/ponder@ponder-0.16.6).
New since 0.16.6-sqd.2
0.16.6-sqd.2 was a 2026-07-08 layer (the correctness wave through the #47 mid-range-204 close, the #50 warmup, and the #20/#94 needed-field scoping). -sqd.3 brings the line onto the current layer, adding:
accessListhonest-NULL fix (#110, #111) — Portal chains that omitaccess_list(e.g. arbitrum/avalanche, an upstream 400) now storeNULLinstead of a fabricated[], byte-identical to geth ground truth.- Realtime body-cap filtered fallback (#23).
- INV-17 realtime-finalize write-side idempotence (#175) — no duplicated factory-child rows on realtime crash/resume.
inserted.logsstore-inserted metric (#143).- Factory log-query byte-budget sharding (#194, #195) and account-tx (from/to) byte-budget sharding (#196, #199) — over-cap IN-list unions split into byte-sized shards, completeness-preserving, byte-identical no-op below the wall.
- INV-26 store-state partition-invariance codification (invariant catalog now INV-1…INV-26).
The full invariant-first re-architecture and earlier correctness wave carry forward unchanged.
Compatibility
- Built + tested green on
ponder@0.16.6: 400/400 across 23 files (CI=true scripts/sync-upstream.sh 0.16.6 --test, pnpm@9.10.0) — verified both by a pre-publish gate and by the release workflow's own in-CI build+test. - The graft seam (
HistoricalSync.syncBlockRangeData/syncBlockData) is untouched; every added fix lives in the Portal layer, off the seam. No fresh RPC byte-diff; compat rests on seam-identity + the full suite.