v4.4.0

Nuthatch 4.4.0 brings a nest's serving process inside its 2 GiB budget under production's real
load, makes kittiwake's one-day QoS statements about seven times faster, and closes the races found
while testing it. It is a minor rather than a patch because memory behaviour changes: the limit
NUTHATCH_BURRMILL_MEMORY_LIMIT now bounds the nest rather than each statement's session, and the
allocator runs with different settings.

Memory

• One pool per nest. Every analytics session over one nest's data now draws on one shared memory
  pool, sized by NUTHATCH_BURRMILL_MEMORY_LIMIT. Before, a statement that arrived while the cached
  session was busy opened a second session with its own pool, so two concurrent statements could
  reserve nearly twice the limit. The second statement now competes for the same budget and spills or
  refuses. Two mounts of one dataset share its pool.
• Huge pages kept off the heap. On a kernel with transparent huge pages set to always, memory
  jemalloc had freed stayed resident inside huge pages. The binary now runs jemalloc with
  thp:never,narenas:8,dirty_decay_ms:1000,background_thread:true. _RJEM_MALLOC_CONF overrides
  it.
• The startup check counts one pool per nest, not SQL permits times the limit, so a configuration
  that fits once is accepted at two permits.

Measured with the release gate on Linux, the allocations nest's 74 production statements, two at a
time as production runs them:

Build	Peak RSS
4.3.1	2,187 to 2,257 MiB
4.4.0	1,595 to 1,788 MiB, four runs

Speed

A hash join whose build side is small and whose condition includes a range now uses that side's
minimum and maximum to skip row groups of the large side's Parquet. kittiwake's one-day statements on
the QoS nest had read every row of a 35 to 79 million row table each time. On a copy of that nest:

Statement	4.3.1	4.4.0
indexer daily	8.4 to 11.9 s	1.2 to 1.3 s
seconds behind	7.2 to 7.3 s	1.1 s
allocation daily	8.1 to 8.3 s	1.1 to 1.2 s

The answers are the same rows in the same order. A semi join produced by an EXISTS or an
IN (subquery) also builds on the smaller side now, so the QoS views' unfiltered counts no longer
refuse out of memory.

Ingestion and runtime

• A seal-direct eth_getLogs answer larger than 32 MiB is refused and the range narrowed, so a node
  with no result cap cannot hand one window more than the budget. A remainder fetch asks only for
  what it needs.
• A sweep follower can no longer miss its leader's wake-up when the leader panics.
• A move whose drain fails backs out its staged mount. If that back-out fails too, the error names
  both and the staged mount keeps its record.
• /explain plans every statement /sql answers, including one that orders by a column it also
  casts under the same name.

Operations

• Parity against the Network Subgraph runs daily on the allocations nest, at the sealed pin and at
  head. A gateway failure pages "PARITY NOT RUN" with the reason. Fee collections that the two sides
  file into neighbouring epochs count as a known difference only when they trace exactly.
• The release gate compares each statement's answer with production's, runs two statements at a
  time, refreshes its copy from Helsinki before each run, and takes production's version from that
  copy.
• deploy/roll-helsinki-from-mac.sh <version> rolls a published release, smoke test first.

Compatibility and upgrade

Replace the binary and restart. There is no storage migration or re-index. An operator who set
NUTHATCH_BURRMILL_MEMORY_LIMIT expecting each statement to get the whole limit now has that limit
shared by the nest's concurrent statements.