Skip to content

v4.10.0

Choose a tag to compare

@github-actions github-actions released this 05 Oct 17:32
· 475 commits to main since this release
7784ace

Nuthatch 4.10.0 makes the engine count the memory a scan holds, so a nest's 2 GiB budget can be met by
its own settings and enforced by its own startup check. It is a minor because of two new budget
settings.

Memory

On kittiwake's QoS nest, up to 1,200 MiB was held outside the engine's memory pool while statements
ran, so the startup check could not fit the 2 GiB per-cursor budget honestly (#1899).

  • A scan no longer keeps a metrics record of about 10 KiB for every segment it opens; its remaining
    per-segment bookkeeping is charged to the pool. On a table of 5,400 segments that was over 300 MiB
    outside the pool.
  • A window's output leaves it in batches of the configured size instead of one batch per partition,
    so an ordered aggregate above it no longer reserves its whole partition at once. The pool QoS needs
    fell from 1.25 GiB to 640 MB.

On a copy of the QoS nest, two statements at a time: peak RSS 1,746 to 1,749 MiB before, 1,298 to
1,349 MiB after, with identical answers. The allocations nest's answers and times are unchanged.

Budget settings

The startup check counts the engine pool, the ingestion reservation and a fixed headroom against
NUTHATCH_MAX_RSS. Two settings let an operator use measured terms instead of the defaults:

  • NUTHATCH_RUNTIME_HEADROOM: the memory the process holds outside the pool, measured.
  • NUTHATCH_INGESTION_RESERVATION may go below 1024 MB, down to 256 MB, but only beside a measured
    headroom. Lowering it alone is still refused.

A refused configuration now names the setting that counted and the largest value that fits.
docs/operators.md says how to measure each term. Unset, both behave as before.

Compatibility and upgrade

Replace the binary and restart. There is no storage migration or re-index. A nest that sets neither
new setting starts exactly as on 4.9.0.