Skip to content

v1.18.0

Choose a tag to compare

@github-actions github-actions released this 13 Oct 19:28
· 14 commits to main since this release
v1.18.0
ba2cbc4

Added

  • --memory-limit (default auto): a soft memory limit that engages only where the tier is
    already predicted not to fit
    (#314). On both
    mount and serve nfs.

    The tier is live Go heap, and at the default GOGC=100 the collector lets the heap grow to
    about twice the live set — measured externally at next_gc = 2.03 × the tier with RSS
    within 1% of it. So a tier over half the budget is an OOM-kill on the way to filling it.
    auto sets a limit at 90% of the budget in exactly that case and changes nothing
    otherwise
    : at the default 25% tier the collector's target is ~50% of the budget, far under
    any limit. Its boundary is the headroom warning's own condition (2 × tier ≥ budget), and a
    test asserts the two agree.

    This reverses #314's first-round ruling, for a stated reason. That round recommended
    GOMEMLIMIT without setting it, because the right value depends on what else is on the box.
    The objection is sound for imposing a limit on every mount and does not survive the case where
    the box is already predicted to OOM, where the comparison is not "0.90 versus 0.85" but "a limit
    under the budget versus the kernel killing the process". The 90% follows the Go team's guidance
    to leave 5–10% headroom for memory the runtime does not account for; it is not measured here.

    A GOMEMLIMIT in the environment is honoured as-is under auto; an explicit size wins over
    both. The limit in force is logged at startup, and a limit at or below --mem-cache warns,
    because the tier is live heap the collector cannot reclaim. Both commands go through one
    applyMemoryLimit, with an AST check that every command sizing a tier applies it — this
    project has seven recorded cases of mount and gateway wiring one mechanism differently.

  • --demand-precise-nontiling: a demand read can be served its extents when the coverage
    signal has judged the handle scattered
    (#408).
    Experimental, off by default, and the reason it is off is the interesting part.

    The finding corrects the diagnosis on the issue. A ZIP read of a 10 MB archive — central
    directory, then 20 random 20 KiB entries — fetches the whole archive, confirmed externally
    against a CloudTrail ledger. The report concluded this was a demand-chunk-size floor and not
    fixable by prefetch policy. The second half is right; the first is not. lith has a byte-exact
    demand lane and these reads are far under its threshold (32 KiB against 1 MiB). What blocked it
    is the lane's predicate, Random || Strided — states denominated in blocks. A 10 MB object
    over 8 MiB blocks is 1.25 blocks, and measured, 20 of 22 reads land in the same block, 1
    adjacent, 0 further. The handle sits in Cold forever. Its trailing coverage meanwhile is
    0.056 against the 0.50 threshold with the window full: the block-space machine has nothing
    to say and the byte-space signal is emphatic.

    So prefetch.NotTiling is a third qualifying claim rather than a loosening of the first two —
    not "we have not decided" and not "the block deltas are wild", but "the bytes this handle
    touched do not tile". It is byte-denominated, so object size cannot defeat it. With it on, the
    ZIP shape fetches 3.3 MB instead of 9.4 MB.

    ★ And it is off by default because it trades bytes for round trips, with the arithmetic against
    it.
    The same shape's fills go 9 → 16: whole-chunk fetching amortizes faults for a
    scattered reader whose entries cluster — 20 entries across 10 chunks means two often share one
    fetch. 6.2 MB saved buys 5–20 ms of transfer; 7 extra round trips cost 210–990 ms at a
    30–141 ms first byte, a loss in every regime. That is the same bytes-for-wall shape as the
    default rolled back one release earlier, caught here before shipping rather than after, by
    reading the GET count in my own test output.

    So the right reading of #408 is not a defect but a deliberate latency-versus-bytes trade,
    which lith currently makes in favour of latency — correctly for most mounts. The flag is for the
    case where egress cost dominates: a cross-region or cross-cloud mount reading archives it
    touches sparsely.

    What it cannot reach, and the guard is pre-existing. A tiling stream has coverage ~1 and
    never matches; TestColdSequentialGetShape, the #233 witness, catches a version that drops the
    window-full requirement, so the safety property was already pinned rather than invented for
    this. And a handle with a committed window cannot match either: the coverage judgement and
    establishment are mutually exclusive by construction, which also means nothing is
    prefetching for a handle this matches, so a precise read cannot land on top of a whole-block
    prefetch — #222's measured-worse case.

  • The sub-block witness now covers request count, the third axis (#407).
    An external bracketed cell on v1.17.1 confirmed the lead closes the coverage gap — uncovered
    misses back to the =false level, 62–65 against 60–62, where span-once gave 81 — and the real
    netCDF4 tool came out neutral. But the index-then-stream witness read a median 0.854, and
    the only remaining difference between arms was ~+41 GETs for identical bytes.

    Reproduced locally: 62 → 79–81 GETs on that shape with bytes unchanged. 41 extra round trips
    at a 30 ms first byte is 1.23 s against a ~7.4 s stream = 17%, which brackets the 11–25% of
    their slow reps.

    So a witness for this arm has to cover bytes, coverage and requests. It covered bytes,
    which is how a 20% regression shipped; then coverage, which is how the lead fix was verified;
    this is the third. The span dispatch count is now asserted rather than the GET count, because it
    is the deterministic driver — 16 in three consecutive runs while GETs came out 79, 81, 79.

    The cause is named and not yet fixed, with the arithmetic for why: 16 dispatches is about 5×
    what this shape needs. The span phase covers only the first ~8 MB of the stream, and across it
    the lead grows from ~4 MB to ~16 MB, so the frontier runs away from the reader and ~3
    dispatches would cover it. The extra ones happen because the frontier is recomputed as
    readEnd + earned on every read and earned grows faster than the reader advances, so the
    comparison against the previous frontier never short-circuits. Fixing it needs a trigger tied to
    the remaining lead rather than to the earned amount — a tuning decision better made from a
    measurement than invented, which is why this release asserts the current count instead.

Fixed

  • Inside a container, lith sized its memory against the host's RAM
    (#314). /proc/meminfo is not namespaced by
    the memory controller, so a container reads the host's MemTotal. A container limited to
    4 GiB on a 256 GiB host therefore defaulted to a 64 GiB memory tier — sixteen times its
    whole allowance — and the startup headroom warning, comparing against the same figure, stayed
    silent. The image's entrypoint is lith serve nfs and a Kubernetes pod with a memory limit is
    the ordinary deployment, so the shipped default was an OOM-kill on the ordinary case.

    The memory budget is now the smaller of MemTotal and the cgroup limit (v2 memory.max,
    v1 memory.limit_in_bytes), with both spellings of "unlimited" recognised rather than parsed —
    v2's literal max and v1's page-rounded near-MaxInt64. The default tier, the headroom warning
    and the new soft limit all read it.

  • TestStreamingByteIdentical's fixture was not a stream. It read 64 KiB at every 1 MiB
    boundary — 1/16 density — and asserted that a "strictly sequential read run" never goes
    byte-exact. Measured, that handle never leaves Cold, never establishes, has window 0 and
    prefetches nothing, and its trailing coverage is 0.0625. The name and the premise described a
    reader the fixture did not contain, and the chunk-alignment assertion was protecting a property
    that reader does not have.

    Found by #408: extending the demand lane took that reader from 16× to 2.4× and the test
    failed — correctly reporting a change, incorrectly calling it a leak. The fixture now reads
    whole chunks, which is what its intent requires, with preconditions asserting the handle
    actually establishes and is judged tiling. The 1/16 case is now its own test, recording that it
    is served precisely and why that is safe there.