Repository navigation
v1.18.0
Added
-
--memory-limit(defaultauto): a soft memory limit that engages only where the tier is
already predicted not to fit (#314). On both
mountandserve nfs.The tier is live Go heap, and at the default
GOGC=100the collector lets the heap grow to
about twice the live set — measured externally atnext_gc= 2.03 × the tier with RSS
within 1% of it. So a tier over half the budget is an OOM-kill on the way to filling it.
autosets a limit at 90% of the budget in exactly that case and changes nothing
otherwise: at the default 25% tier the collector's target is ~50% of the budget, far under
any limit. Its boundary is the headroom warning's own condition (2 × tier ≥ budget), and a
test asserts the two agree.This reverses #314's first-round ruling, for a stated reason. That round recommended
GOMEMLIMITwithout setting it, because the right value depends on what else is on the box.
The objection is sound for imposing a limit on every mount and does not survive the case where
the box is already predicted to OOM, where the comparison is not "0.90 versus 0.85" but "a limit
under the budget versus the kernel killing the process". The 90% follows the Go team's guidance
to leave 5–10% headroom for memory the runtime does not account for; it is not measured here.A
GOMEMLIMITin the environment is honoured as-is underauto; an explicit size wins over
both. The limit in force is logged at startup, and a limit at or below--mem-cachewarns,
because the tier is live heap the collector cannot reclaim. Both commands go through one
applyMemoryLimit, with an AST check that every command sizing a tier applies it — this
project has seven recorded cases of mount and gateway wiring one mechanism differently. -
--demand-precise-nontiling: a demand read can be served its extents when the coverage
signal has judged the handle scattered (#408).
Experimental, off by default, and the reason it is off is the interesting part.The finding corrects the diagnosis on the issue. A ZIP read of a 10 MB archive — central
directory, then 20 random 20 KiB entries — fetches the whole archive, confirmed externally
against a CloudTrail ledger. The report concluded this was a demand-chunk-size floor and not
fixable by prefetch policy. The second half is right; the first is not. lith has a byte-exact
demand lane and these reads are far under its threshold (32 KiB against 1 MiB). What blocked it
is the lane's predicate,Random || Strided— states denominated in blocks. A 10 MB object
over 8 MiB blocks is 1.25 blocks, and measured, 20 of 22 reads land in the same block, 1
adjacent, 0 further. The handle sits inColdforever. Its trailing coverage meanwhile is
0.056 against the 0.50 threshold with the window full: the block-space machine has nothing
to say and the byte-space signal is emphatic.So
prefetch.NotTilingis a third qualifying claim rather than a loosening of the first two —
not "we have not decided" and not "the block deltas are wild", but "the bytes this handle
touched do not tile". It is byte-denominated, so object size cannot defeat it. With it on, the
ZIP shape fetches 3.3 MB instead of 9.4 MB.★ And it is off by default because it trades bytes for round trips, with the arithmetic against
it. The same shape's fills go 9 → 16: whole-chunk fetching amortizes faults for a
scattered reader whose entries cluster — 20 entries across 10 chunks means two often share one
fetch. 6.2 MB saved buys 5–20 ms of transfer; 7 extra round trips cost 210–990 ms at a
30–141 ms first byte, a loss in every regime. That is the same bytes-for-wall shape as the
default rolled back one release earlier, caught here before shipping rather than after, by
reading the GET count in my own test output.So the right reading of #408 is not a defect but a deliberate latency-versus-bytes trade,
which lith currently makes in favour of latency — correctly for most mounts. The flag is for the
case where egress cost dominates: a cross-region or cross-cloud mount reading archives it
touches sparsely.What it cannot reach, and the guard is pre-existing. A tiling stream has coverage ~1 and
never matches;TestColdSequentialGetShape, the #233 witness, catches a version that drops the
window-full requirement, so the safety property was already pinned rather than invented for
this. And a handle with a committed window cannot match either: the coverage judgement and
establishment are mutually exclusive by construction, which also means nothing is
prefetching for a handle this matches, so a precise read cannot land on top of a whole-block
prefetch — #222's measured-worse case. -
The sub-block witness now covers request count, the third axis (#407).
An external bracketed cell on v1.17.1 confirmed the lead closes the coverage gap — uncovered
misses back to the=falselevel, 62–65 against 60–62, where span-once gave 81 — and the real
netCDF4 tool came out neutral. But the index-then-stream witness read a median 0.854, and
the only remaining difference between arms was ~+41 GETs for identical bytes.Reproduced locally: 62 → 79–81 GETs on that shape with bytes unchanged. 41 extra round trips
at a 30 ms first byte is 1.23 s against a ~7.4 s stream = 17%, which brackets the 11–25% of
their slow reps.So a witness for this arm has to cover bytes, coverage and requests. It covered bytes,
which is how a 20% regression shipped; then coverage, which is how the lead fix was verified;
this is the third. The span dispatch count is now asserted rather than the GET count, because it
is the deterministic driver — 16 in three consecutive runs while GETs came out 79, 81, 79.The cause is named and not yet fixed, with the arithmetic for why: 16 dispatches is about 5×
what this shape needs. The span phase covers only the first ~8 MB of the stream, and across it
the lead grows from ~4 MB to ~16 MB, so the frontier runs away from the reader and ~3
dispatches would cover it. The extra ones happen because the frontier is recomputed as
readEnd + earnedon every read andearnedgrows faster than the reader advances, so the
comparison against the previous frontier never short-circuits. Fixing it needs a trigger tied to
the remaining lead rather than to the earned amount — a tuning decision better made from a
measurement than invented, which is why this release asserts the current count instead.
Fixed
-
Inside a container, lith sized its memory against the host's RAM
(#314)./proc/meminfois not namespaced by
the memory controller, so a container reads the host'sMemTotal. A container limited to
4 GiB on a 256 GiB host therefore defaulted to a 64 GiB memory tier — sixteen times its
whole allowance — and the startup headroom warning, comparing against the same figure, stayed
silent. The image's entrypoint islith serve nfsand a Kubernetes pod with a memory limit is
the ordinary deployment, so the shipped default was an OOM-kill on the ordinary case.The memory budget is now the smaller of
MemTotaland the cgroup limit (v2memory.max,
v1memory.limit_in_bytes), with both spellings of "unlimited" recognised rather than parsed —
v2's literalmaxand v1's page-rounded near-MaxInt64. The default tier, the headroom warning
and the new soft limit all read it. -
TestStreamingByteIdentical's fixture was not a stream. It read 64 KiB at every 1 MiB
boundary — 1/16 density — and asserted that a "strictly sequential read run" never goes
byte-exact. Measured, that handle never leavesCold, never establishes, has window 0 and
prefetches nothing, and its trailing coverage is 0.0625. The name and the premise described a
reader the fixture did not contain, and the chunk-alignment assertion was protecting a property
that reader does not have.Found by #408: extending the demand lane took that reader from 16× to 2.4× and the test
failed — correctly reporting a change, incorrectly calling it a leak. The fixture now reads
whole chunks, which is what its intent requires, with preconditions asserting the handle
actually establishes and is judged tiling. The 1/16 case is now its own test, recording that it
is served precisely and why that is safe there.