Repository navigation
v1.14.0
Added
-
--readahead-subblock-dispatch: a readahead dispatch can be shorter than one block
(#407, arm B). Experimental, off by default.The evidence gate is denominated in blocks and floors at 2, so the smallest window any
established handle can be given is 16 MiB at the default 8 MiB block — while a GRIB.idx
field sweep, an HDF5 chunk walk and a FITS cutout's row bands all read 0.5–2 MB per run.
This converts that residual from blocks into bytes: a handle that has earned less than the
floor is dispatched a contiguous span of exactly what it earned, placed immediately after the
read that earned it, and fetched through the same extent path a strided prediction uses.This is what makes
--readahead-evidence-per-runable to do anything. That arm shipped in
v1.13.0 and an external sweep then measured it identical to the default on all three walk
shapes. The null was structural — arm A correctly shrinks the earned window and the block floor
hands the same dispatch straight back. Measured locally on a 7-field walk fixture, dispatched
bytes over the 2×2:arm B off arm B on per-run off baseline decays to 1.03× per-run on 1.00× (exactly inert) 12.1× Neither arm alone holds up, and they fail for different reasons — which is why they are two
flags rather than one default. Arm A is exactly inert, to the byte. Arm B alone is not inert
but decays: it reads lifetime consumption, which grows past the floor's worth within a few
runs, after which it switches itself off — its span count pins at 13 whether the walk is 8
fields or 64, while the pair issues one per landing indefinitely. This is #222's
precedent repeating, where each half alone made bytes worse and only both together gave 16×.End to end against a fake S3, the pair takes a walk from 7.20× to 3.55× over-fetch —
2.03× fewer bytes — moving fills fromkind="whole"tokind="prefetch-range", which is
how the change is visible from outside.It cannot reach a streaming reader, and that is a precondition rather than a threshold. The
arm engages only for a handle that has finished an established run, which a sequential copy
never does. Today the 2-block floor is what protects that reader, so an arm that removes the
floor has to carry the protection itself — #233
says a cold sequential read should fetch more, and #231
refuted three attempts to make it fetch sooner.
Fixed
-
--readahead-evidence-per-run's help motivated it with figures from a ratio that is no
longer the default (#407). The flag's help
and itsknobs.mdrow quoted "48 chunks prefetched and 2 used on GRIB ... a peak window of
5 blocks", all measured at ratio 4 — and v1.13.0 moved the in-region default to 2, at
which point the flag does nothing at all on that shape. 11.9 MB at ratio 2 over an 8 MiB
block earns 2.84 blocks, which floors to the same 2 blocks a 0.5–2 MB run earns, so both
bases return the identical window.Caught by an external null result that was structural rather than informative. A sweep on
the v1.13.0 artifact measured this flag identical to the default on all three walk shapes and
recommended retiring it. The arms were identical by arithmetic on that shape, so the cell
had no power to separate them — the same reading that made ratio 1's tie with ratio 2
uninformative, pointed the other way. The flag is kept.Discrimination starts at a lifetime consumption of
3 × block ÷ ratio= 12.6 MB at the
defaults, and the measured shape sits 5.2% below it. Above the boundary the cumulative
basis grows without bound — 4 blocks at 20 MB, 23 at 100 MB — while a short run stays pinned
at 2. Both the help and the row now state the boundary, and a test pins it, because the only
test of this arm's mechanism ran at ratio 4 and so kept passing while the default moved out
from under it. -
A finished run was not counted, so arm B was inert on every shape.
deEstablishcleared
establishedbefore callingendRun, andendRunreads that flag to decide whether there
was a run to end at all — so it always sawfalse, no run was ever counted, and the
precondition above could never be met. Found by instrumenting a walk fixture that
re-established four times and reported zero finished runs; every decision-level test for the
arm was green while it could not fire end to end, which is thettfbRecordershape again. -
The two-block dispatch floor was documented as 17 MiB; it is 16 MiB. 2 × 8 MiB is 16 MiB
(≈17 MB), and the wrong unit had propagated into--readahead-evidence-per-run's help,
docs/knobs.mdand two places in v1.13.0's changelog section. Corrected in all four,
including the released section — a wrong figure where people read it is worse than an
edited historical entry — so that section now differs from the v1.13.0 release body as
published. Same class as #340's 5 ms, which was the wrong unit rather than the wrong value.