Skip to content

v3.0.0

Choose a tag to compare

@github-actions github-actions released this 29 Aug 09:05
· 19 commits to main since this release
c081cae

Authored incremental entities, stable. A nest declares a relation in entities.toml and nuthatch
maintains it as blocks arrive instead of recomputing it on every query. That is RFC-0041, and it is
what the major bump is about.

3.0.0-alpha.1 carried the feature; this carries it plus what a day of soaking it on real chains
found.

Drop-in, with one sentence of small print

Replace the binary and restart. No config migration, no data migration, no re-index. A nest that
declares no entities behaves exactly as it did on 2.7.x. Verified on production data: all 73 tables of
the Lodestar Horizon nest re-counted at a pinned watermark before and after the swap, identical,
2,376,135 rows
.

The small print: adding an entities.toml to a nest that already has history means its next restart
seeds the entity from the whole sealed corpus.
Measured rather than warned about: 249,979 rows across
733 sealed segments in 1.9 s, and a larger Horizon corpus of 346,288 rows across 2,985 segments in
2.4 s. Independently corroborated on macOS at 2 s, including after a SIGKILL.

What an entity buys

The Lodestar indexer_rewards panel, which was an authored view over 733 sealed segments and is now a
maintained relation. Same nest, same data, same query, p50 over 25 runs:

p50 p99
authored view, recomputed per query 2.15 s 2.20 s
maintained relation 87.7 ms 92.1 ms

Identical rows, all 82 of them. One block's update against a relation of 309,548 groups is 285 µs,
against 61 groups it is 195 - flat, which is the point, and not true until late in the work.

Verify who built your binary

New, and the reason to bother is that a .sha256 sidecar downloaded beside the tarball it describes
proves only that the file did not corrupt in transit. Anyone able to replace one could replace the
other in the same breath.

gh attestation verify nuthatch-x86_64-unknown-linux-gnu.tar.gz --repo nightswatchhq/nuthatch

Every release artifact now carries a build provenance attestation, signed by GitHub's identity for the
workflow run that produced it and recorded in a public transparency log: which repository, which
commit, which workflow. --repo is load-bearing - without it an attestation from any repository is
accepted. The release workflow runs the same check against its own assets before publishing, so a
release whose provenance does not verify never goes public.

Every third-party Action in the build is also pinned to an immutable commit SHA rather than a movable
tag, because provenance signed by a workflow running whatever a tag was retagged to proves rather less
than it appears to.

Five defects the soak found that the test suite could not

Each needed a real nest and real chains.

A pool-wide 429 killed a nest with block N alone exceeds the provider's getLogs result cap - a
diagnosis that was never true, whose stated remedy (a bigger provider) does nothing about throttling.
Every endpoint refusing one request was escalated to "narrow it", the chunker halved to a single block,
and there was nothing left to narrow. Found on a production box crash-looping twice an hour.

An answer could be labelled more current than its own rows. /sql executed against a maintained
relation and then read applied_through afresh - two lock acquisitions - so a batch landing between
them produced rows from block N carrying the label N+8. 1 in 12 reads on a 0.25s-block chain. The
relation was never wrong; the citation was.

A backfill that stopped advancing ran forever. It now fails by name after 64 attempts without
progress. At tip it warns instead, because a tip loop faulting on somebody else's outage turns a
transient problem into a dead nest.

A restart silently kept only groups mod 10,000 of a relation (fixed in the alpha), and
one block's update cost 72 ms and grew with history (likewise). Both invisible from CI.

Also in here

--seal-direct refuses a nest declaring an entity rather than completing with an empty relation. Six
Prometheus series per entity: applied_through, current, rows, faulted, unavailable,
seconds_since_progress. [[alerts]] gains format = "discord" for sinks wanting {"content": …};
the default is unchanged and every existing sink is byte-identical.

A /sql request was defining a view for every table in the nest's manifest whether the statement named
it or not, at roughly 62 µs per sealed segment - SELECT 1 cost 2.5 s on a 38,428-segment nest
before reading a row. Views are now defined for what a statement can reach. degraded_tables narrowed
with it: a query reports degraded segments for tables it read, and the nest-wide sweep runs
separately rather than on the caller's time. That is the one behaviour change on an existing surface.

What has been soaked, and what has not

An entity followed Arbitrum for over 24 hours without once falling behind its head, checked against a
full recompute at the same block every hour: exact every time. A parallel Ethereum nest took a real
reorg and converged. Six restarts alternating SIGKILL and SIGTERM on a live nest, exact after each.

Not yet soaked: a process carrying an entity restarted after days rather than hours, and the two
per-cursor RSS admission constants remain one measurement on one machine.
docs/releases/3.0.0-alpha-stresstest.md says what is still open and in what order it gets answered.