v3.5.0
The release that turns the RPC bill down. Every nest following a chain tip should restart onto this
with a --poll-interval matched to how often anyone reads it; nothing changes on disk.
The freshness dial: --poll-interval and --finality-only (#1173, RFC-0040)
Measured on a nest at tip against a paid endpoint on 2026-09-06: 268 block headers, 61 log
requests and 84 tip polls a minute, about 9,900 compute units a minute, roughly $185 a month. Of
that day's 345,600 Arbitrum blocks, 95 carried an event the nest wanted. The code already bought
headers only for those; the rest was the poll loop's own - a reorg check on every two-second poll,
a checkpoint hash and a finalized probe on every committed window. The bill was the cadence, not
the data.
--poll-interval <DURATION> (2s default; 5m, 1h, or bare seconds) is how long a caught-up
cursor waits before asking for the tip again. At five minutes the same nest spends on the order of
a hundred compute units a minute for the same rows. --finality-only caps the cursor at the
chain's finality boundary instead of the tip: nothing it indexes can be reorged, so the reorg
check stops once the hot store holds no unfinalised row, and the hot store only ever carries rows
waiting to seal. Both are nuthatch dev flags on the solo and the runtime path, never
nuthatch.toml fields: how often an operator asks is not what the nest is, and two nests differing
only in cadence hold identical rows under one content address.
Nothing about it is silent. /ready carries "freshness": {"mode": "tip" | "finality", "poll_interval_secs": N}, lag_blocks is then the distance the operator chose, and the stall
thresholds behind ready scale to at least three intervals, so a quiet five-minute cursor is not
reported dead and a dead pool is still reported inside a quarter of an hour. Sealing is untouched.
A window lowered by rate limiting recovers (#1170)
A pool-wide 429 is treated as a refusal to narrow, so the fetcher halves the range until the pieces
are served, and the controller lowered its ceiling to the width that came back. Nothing ever raised
it: an hour of 429s from a co-tenant walked a backfill's window down to about ten blocks and it ran a
hundred times slower for the next three hours until an operator restarted it. The ceiling now
recovers - four consecutive windows served whole at exactly the lowered ceiling double it, bounded
by the ceiling the controller was built with, and any refusal resets the count. Wider windows that
were already in flight when the refusal landed do not count; they are evidence about the provider
before the pressure, not after. The live window is on /ready as fetch_window_blocks and on
/metrics as nuthatch_fetch_window_blocks, so a collapsed one is visible before its ETA is.
/ready tells fetched from sealed during a direct-seal backfill (#1169)
seal_direct_completed was set by the progress callback on every fetched window, so /ready,
nuthatch_seal_direct_completed and every "done N of M" figure read from them were how far
fetching had got. The durable watermark a restart resumes from is written when a segment lands
on disk, and on a sparse range that lagged fetching by 47,599,977 blocks on the gns nest; the restart
redid all of them. Two counters now: seal_direct_completed moves only when a segment is durable,
and seal_direct_fetched (new, on /ready and /metrics) is the fetch position, a high-water
mark that cannot move backwards. The gap between them is exactly the work a restart redoes; bounding
it is #1178.
The DuckDB memory budget is shared by the permits (#1165)
Each analytical permit opened its own engine with a 512 MB ceiling, so four permits could claim
2 GB and a burst of dashboard queries drove a nest into the kernel's page-allocation failure.
The permits now share one budget: min(512 MB, 1024 MB / permits), with the divisor clamped to the
gate's ceiling so the arithmetic holds for any permit count.
Also
serveno longer refuses a nest that declares[[calls]]for lacking--state-rpc: that role
samples no calls (#1167).- Operator docs: the measured bill and the dial are in
docs/operators.md; RFC-0040 §6 records
what the measurement said about the RFC's own ordering.