Linux auto-budget can overestimate usable memory under host pressure or cgroup limits

[mccoyspace]

Problem

When ram_budget_bytes == 0, WASTE currently caps its automatic choice at 7/8 of installed physical RAM. On Linux, that can be larger than the memory the process can safely allocate when the host already has resident workloads or a cgroup-v2 ancestor imposes a lower limit.

This is the Linux/open-time counterpart to the paging cliff already documented in docs/LEARNED.md §16: the physical cap remains useful, but it is not sufficient when current host pressure or a finite cgroup is tighter.

This came out of the GN100/K3 investigation, with one important correction: the original 80.64 GiB paging sample was caused by a second WASTE model open after budget selection. Clean 80.64 GiB replays were safe. This proposal protects against pressure already present when waste_open begins; it cannot predict a workload that starts later.

Proposed policy

For automatic budgets only, bound the ceiling by the minimum of:

• the existing 7/8 physical-RAM cap;
• Linux MemAvailable minus a 1/8 host-RAM reserve; and
• finite cgroup-v2 ancestor headroom (memory.max - memory.current) minus 1/8 of that group effective capacity.

The cgroup reserve is based on min(memory.max, physical RAM), rather than host RAM alone. Otherwise an 8 GiB group on a 128 GiB host incorrectly loses a 16 GiB reserve. The reader walks ancestors so an unlimited leaf cannot hide a finite parent.

If a known live/cgroup ceiling cannot hold the model floor, the candidate fails before model-sized allocation. This deliberately differs from the current behavior of running at the floor and warning, so I would value guidance on that choice.

Explicit nonzero budgets remain caller policy: they are not silently clamped, but receive a warning above the current safe ceiling. Missing or malformed Linux telemetry falls back safely, non-Linux behavior remains unchanged, and cgroup v1 is deliberately out of scope.

Tested candidate

I have this isolated against current main in candidate commit 5e4147f.

Validation completed:

• policy and synthetic proc/cgroup tests cover malformed telemetry, exhausted limits, an 8 GiB child, an unlimited child under a finite parent, ancestor traversal, whole-working-set selection, and floor refusal;
• macOS portable suite: 29 passed, 0 failed, 12 skipped; Python server: 168 passed;
• ASan/UBSan: 28 passed, 0 failed, 13 skipped;
• GN100 Linux ARM64: 27 passed, 0 failed, 13 skipped;
• a real K3 automatic open on an otherwise idle GN100 retained the full recommended 86,583,021,568-byte budget, including a 55,780,900,864-byte expert cache, and completed normally in 24.1 seconds.

This is a safety change, not a throughput claim.

Before I open a PR, do these policy boundaries look right? In particular, should explicit budgets remain authoritative, should auto-open refuse when the current ceiling is below the floor, and is the proposed public waste_memory_ceiling() query plus JSON reporting useful? I can narrow or reshape the patch before opening it.

[marcobambini]

You found a real bug, and thank you for correcting the 80.64 GiB sample rather than
shipping it as evidence — that changed how I read the proposal. What was left is not
"we measured paging", it is "physical RAM is the wrong question on Linux", which is
the better argument and the one that holds.

I have taken the capacity half: #15.
It is shaped differently from your candidate and I want to be explicit about why,
because the reshaping is also my answer to your three questions.

What went in

waste_physical_ram() on Linux is sysconf(_SC_PHYS_PAGES), which reads the host's
MemTotal and reads exactly the same thing from inside a cgroup allowed a fraction
of it. So the ceiling is now min(physical, cgroup limit) — the smallest finite
memory.max or memory.high across the cgroup and its ancestors — and
everything downstream is untouched: the 7/8 reserve, the whole-working-set stepping,
the floor refusal, every non-Linux host.

memory.high earns its place for the reason §16 exists: a group the kernel reclaims
from is a group whose expert cache it takes back, and a hit that costs a page fault
is worse than the pread it replaced. That is your argument, arriving by another road.

Measured in a container, which is the only place it can be measured. docker run --memory=6g on a host reporting 8,319,213,568 bytes gives usable_ram_bytes
6,442,450,944 — the limit exactly — and the suite's own budget check inside that
cgroup reports usable 6.00 GB, which is the resolver consuming it rather than the
reader merely reading it. 24 passed, 0 failed on Linux there. The choice changes and
not only the reading: a 12 MiB cgroup holds floor + 3x and opens silently, a 9 MiB
one puts the ceiling under the floor so the engine runs at the floor and says so.

Your instinct about the cgroup path was right and I had the shape wrong. I wrote
the ancestor walk for the case where /proc/self/cgroup names a path that does not
exist under the mount. Running it found three shapes, not one: docker run's default
private namespace is 0::/ with the limit in the root; --cgroupns=host reads
0::/docker/<id> and that path does exist, because the host hierarchy is mounted,
while the root has no memory.max at all; only a runtime mounting the leaf alone
gives the shape I had assumed. Reading every level from the composed path up to and
including the root covers all three without having to know which one it is. Your
candidate stopped at the composed path's ancestors and would have been fine on two of
the three — but it also computed headroom from memory.current, and I could not tell
from the tests which case had actually been exercised, because synthetic files
construct a consistent pair and a real container does not.

Your three questions

Explicit budgets stay authoritative. Unchanged, and waste.h already called it a
caller contract. Warn, never clamp.

Auto-open does not refuse. Nothing new refuses, and no new error code went in.
budget < floor_bytes → WASTE_E_RAM_BUDGET already meant "this machine cannot hold
the floor", and it now means it in a cgroup too. Feeding the limit in as capacity
rather than layering a second policy on top is what let that stay true — and it is
also what kept the behavior identical on macOS and Windows, so there is no
#if __linux__ in the reader at all. Nothing it does is a Linux API; guarding it
would mean the one platform that runs it is the one platform that never compiles it
anywhere else, which is §27, and the test would only ever run where nobody develops.

Yes to the query, as waste_usable_ram(), and waste plan --json reports
usable_ram_bytes beside physical_ram_bytes. Without it a JSON reader in a
container computes a ceiling the engine never used and concludes the engine ignored
its own rule.

What is still open, and it is yours if you want it

MemAvailable is not in, and this issue stays open for it. My objection is not that
it is useless, it is that it is a different kind of number. Capacity is fixed for
the life of the process; pressure moves between the read and the allocation. A budget
is resolved once at waste_open and held for a run that lasts hours, so bounding it
by an instantaneous sample makes the same command on the same machine two different
runs — and a host that is busy now says nothing about a host that will be busy in
ten minutes.

That argues against pressure as a ceiling. It does not argue against pressure as
an input to the multiplier, which is already a soft, coarse, reversible choice
between floor + 3x / 2x / 1x / floor. Stepping down one multiple because the host
is loaded costs a fraction of the cache and cannot refuse anything. That is the
version I would review.

Three things I would want in it:

1. The reserve taken once. Your candidate had the 7/8 cap keep an eighth and then
   MemAvailable - physical/8 keep it again. On an idle Linux box MemAvailable runs
   near 95% of MemTotal, so the effective ceiling drops from 0.875x to about 0.825x
   with no measurement behind it — enough to move a machine down a whole working set
   near a boundary, on hosts under no pressure at all.
2. memory.current is not headroom. It includes reclaimable page cache charged to
   the group, so max - current understates what is available, and understates it
   most right after the container has touched files. memory.stat's inactive_file
   and slab_reclaimable are the correction.
3. Do not put a live figure in tests/run.sh's default_budget. It shells out to
   waste plan and waste info as two separate processes. A capacity is the same in
   both; a pressure reading is not, and near a step boundary on a busy machine the
   check fails as an engine bug. This is why Size the automatic budget against the cgroup limit, not the host's RAM #15 asserts against usable_ram_bytes
   and not against anything that moves.

And one measurement would settle more than any of it: on a host under real pressure,
does a budget one multiple smaller actually run faster than the one the current rule
picks? §16 is that curve for physical RAM and it is the sharpest number in the file.
The equivalent row for MemAvailable does not exist yet, and with the router
lookahead in (§39 — a 3.32 GB cache now measures within 10% of a 17.32 GB one) the
cost of stepping down is much lower than it was when §16 was written. The safety
argument no longer has to fight the speed argument, which is the strongest thing your
proposal has going for it and it is not in the proposal yet.

[mccoyspace]

We ran the requested measurement; the step-down preserved headroom but did not improve performance.

[marcobambini]

@mccoyspace thanks for testing!