Skip to content

v3.4.1

Choose a tag to compare

@github-actions github-actions released this 05 Sep 19:38
· 157 commits to main since this release
2c8c594

Three things found on one afternoon of running a fresh 3.4.0 nest behind a live dashboard. Every
nest serving queries should restart onto this; nothing changes on disk.

A query no longer fails when a seal replaces a segment under it (#1162)

A seal folds a provisional segment: the new file is written, the manifest installed, the old file
removed. A query that planned against the old manifest a moment earlier was still holding the old
name, and DuckDB failed it at prepare (No files found that match the pattern) or at execution
(Cannot open file). Three of about forty queries did this in one ten-minute window of a backfill.
The query runner now recognises that shape - a path under segments/ with one of those two
wordings - and plans once more against the manifest as it now is, under the same deadline. Once
only; a second such failure inside one query is reported as before. Nothing is corrupt and nothing is
missing in this case, so the integrity sweep that follows a failed execution is not what answers it.

/ready reports what a direct seal sealed (#1163)

sealed_through on /ready and nuthatch_sealed_through read a gauge that the nest seeds from the
store when it starts and that only the tip-follower's own seal advanced. A direct-seal backfill
wrote its watermark to the store and never to the gauge, so a nest with 460M directly sealed blocks
reported sealed_through 0 until the finalized head crossed its hot store's start, hours later; an
operator's switch script read that as an unfit nest. Both of the direct seal's writers now go
through one seam that writes the store key first and then the gauges, so the gauges never claim a
watermark that was not persisted.

DuckDB's compressed materialisation is off in every build (#1165)

A 3.4.0 nest serving a dashboard's curator lists died twice at the same instruction, a free on
the HTTP response path, which is where heap corruption by native code surfaces. 3.4.0 already
switched this optimiser off in debug builds (#1152), where DuckDB's own assertion aborts on exactly
the shape those lists are: a filtered ORDER BY over a view whose scan reads one segment holding
rows on both sides of the filter. The release library ran past that assertion. It no longer runs the
optimiser at all. This is not proven to be the cause: a fresh process on the same binary and
data took 331 of the dashboard's requests without a fault, and the two crashes came from a process
with a long backfill behind it. It is the cheapest suspect to remove, the answers were already
measured not to change without it, and the proof either way is this release serving the same
pattern for a day.

Also

  • serve refuses a nest that declares [[calls]] and tells the operator to pass --state-rpc,
    which serve does not accept. Recorded, not fixed here; run such a nest with dev.
  • A manual CI job (symbolise.yml) rebuilds a shipped release with symbols kept and names the
    function at a fault offset from a kernel log line, since release binaries are stripped.