Skip to content

v1.14.0

Choose a tag to compare

@github-actions github-actions released this 11 Oct 22:04
· 29 commits to main since this release
v1.14.0
9d9d3d1

Added

  • --readahead-subblock-dispatch: a readahead dispatch can be shorter than one block
    (#407, arm B). Experimental, off by default.

    The evidence gate is denominated in blocks and floors at 2, so the smallest window any
    established handle can be given is 16 MiB at the default 8 MiB block — while a GRIB .idx
    field sweep, an HDF5 chunk walk and a FITS cutout's row bands all read 0.5–2 MB per run.
    This converts that residual from blocks into bytes: a handle that has earned less than the
    floor is dispatched a contiguous span of exactly what it earned, placed immediately after the
    read that earned it, and fetched through the same extent path a strided prediction uses.

    This is what makes --readahead-evidence-per-run able to do anything. That arm shipped in
    v1.13.0 and an external sweep then measured it identical to the default on all three walk
    shapes. The null was structural — arm A correctly shrinks the earned window and the block floor
    hands the same dispatch straight back. Measured locally on a 7-field walk fixture, dispatched
    bytes over the 2×2:

    arm B off arm B on
    per-run off baseline decays to 1.03×
    per-run on 1.00× (exactly inert) 12.1×

    Neither arm alone holds up, and they fail for different reasons — which is why they are two
    flags rather than one default. Arm A is exactly inert, to the byte. Arm B alone is not inert
    but decays: it reads lifetime consumption, which grows past the floor's worth within a few
    runs, after which it switches itself off — its span count pins at 13 whether the walk is 8
    fields or 64, while the pair issues one per landing indefinitely. This is #222's
    precedent repeating, where each half alone made bytes worse and only both together gave 16×.

    End to end against a fake S3, the pair takes a walk from 7.20× to 3.55× over-fetch —
    2.03× fewer bytes — moving fills from kind="whole" to kind="prefetch-range", which is
    how the change is visible from outside.

    It cannot reach a streaming reader, and that is a precondition rather than a threshold. The
    arm engages only for a handle that has finished an established run, which a sequential copy
    never does. Today the 2-block floor is what protects that reader, so an arm that removes the
    floor has to carry the protection itself — #233
    says a cold sequential read should fetch more, and #231
    refuted three attempts to make it fetch sooner.

Fixed

  • --readahead-evidence-per-run's help motivated it with figures from a ratio that is no
    longer the default
    (#407). The flag's help
    and its knobs.md row quoted "48 chunks prefetched and 2 used on GRIB ... a peak window of
    5 blocks", all measured at ratio 4 — and v1.13.0 moved the in-region default to 2, at
    which point the flag does nothing at all on that shape. 11.9 MB at ratio 2 over an 8 MiB
    block earns 2.84 blocks, which floors to the same 2 blocks a 0.5–2 MB run earns, so both
    bases return the identical window.

    Caught by an external null result that was structural rather than informative. A sweep on
    the v1.13.0 artifact measured this flag identical to the default on all three walk shapes and
    recommended retiring it. The arms were identical by arithmetic on that shape, so the cell
    had no power to separate them — the same reading that made ratio 1's tie with ratio 2
    uninformative, pointed the other way. The flag is kept.

    Discrimination starts at a lifetime consumption of 3 × block ÷ ratio = 12.6 MB at the
    defaults, and the measured shape sits 5.2% below it. Above the boundary the cumulative
    basis grows without bound — 4 blocks at 20 MB, 23 at 100 MB — while a short run stays pinned
    at 2. Both the help and the row now state the boundary, and a test pins it, because the only
    test of this arm's mechanism ran at ratio 4 and so kept passing while the default moved out
    from under it.

  • A finished run was not counted, so arm B was inert on every shape. deEstablish cleared
    established before calling endRun, and endRun reads that flag to decide whether there
    was a run to end at all — so it always saw false, no run was ever counted, and the
    precondition above could never be met. Found by instrumenting a walk fixture that
    re-established four times and reported zero finished runs; every decision-level test for the
    arm was green while it could not fire end to end
    , which is the ttfbRecorder shape again.

  • The two-block dispatch floor was documented as 17 MiB; it is 16 MiB. 2 × 8 MiB is 16 MiB
    (≈17 MB), and the wrong unit had propagated into --readahead-evidence-per-run's help,
    docs/knobs.md and two places in v1.13.0's changelog section. Corrected in all four,
    including the released section — a wrong figure where people read it is worse than an
    edited historical entry — so that section now differs from the v1.13.0 release body as
    published. Same class as #340's 5 ms, which was the wrong unit rather than the wrong value.