Skip to content

Two auto-budget opens on one machine each size against the whole machine #31

Description

@marcobambini

waste_open resolves the automatic budget against waste_usable_ram()
capacity, correct as far as it goes — and nothing tells it that another
process has already claimed most of that capacity. Two --budget-less K3
opens on this 64 GB laptop each resolve floor + 1x:

waste: no --budget, using 46.25 GB of 64.00 GB (expert cache 17.56 GB)

Twice, for 92.5 GB asked of a 64 GB machine, and neither open is wrong on its
own terms. This is the case #14
explicitly carved out — its original 80.64 GiB paging sample turned out to be
"a second WASTE model open after budget selection", and the policy that landed
in #15 says so directly: it bounds
pressure already present when waste_open begins and cannot see a workload
that starts later.

What is already decided, and is not up for re-litigation here

  • Capacity is solved. waste_usable_ram() is min(physical, cgroup limit)
    across memory.max/memory.high and ancestors (CHANGELOG 0.6.5,
    docs/LEARNED.md §40).
  • Bounding the ceiling by an instantaneous sample is refuted, and the
    reasoning is in src/memory.c: MemAvailable and memory.current move
    between the read and the allocation, and a budget resolved once and held for
    a whole run "cannot track it. Bounding a long-lived allocation by an
    instantaneous reading makes the same command on the same machine two
    different runs." A proposal that reintroduces that is already answered.
  • Explicit budgets stay authoritative. Warn, never clamp. Whatever comes
    out of this touches ram_budget_bytes == 0 only.

What the 0.6.5 entry left open is the narrower question, quoted from it:
"Whether it should trim the working-set multiplier instead is #14, still
open."
This issue is that question, plus the second data point that
PR #29 supplied.

Why container identity is the wrong key

PR #29 proposes a per-container flock, so that a second process opening the
same container is refused. It is aimed at this problem and misses it in both
directions:

oversubscribes caught by a container lock
two K3 opens, same container yes, 92.5 GB of 64 GB yes
K3 + Kimi-Linear, different containers yes, ~48 GB on top of 46.25 no
two Kimi-Linear opens on 128 GB no yes — refused for nothing
WASTE next to any other memory hog yes no

The inode is a proxy for the hazard, and it is wrong on three of four rows.
The predicate that actually predicts the harm is resident bytes against a
ceiling somebody else has already spent — which is arithmetic waste_open
is already doing, one input short.

Gates, cheapest answer first

Nothing here needs a line of code until gate 1 has a number.

  1. Reproduce the collapse deliberately. Two concurrent auto-budget K3
    opens on the 64 GB machine, decode both, record tok/s and peak RSS for each
    against a single-process baseline of 0.32 tok/s. §16 measured 8x for 29.1 GB
    of cache against 28.0 GB on this machine, so the expectation is that both
    processes end up far below what one process at the floor would do — but the
    two-process case has never been measured on purpose, only inferred from a
    sample that was being used to argue something else. If the OS degrades
    gracefully here, this closes with a dated LEARNED.md entry and PR Prevent competing processes from opening one container #29's
    premise goes with it.

  2. Would accounting have changed the outcome? Same pair, second process
    pinned to --budget floor by hand. If two processes at the floor are
    collectively meaningfully faster than two at floor + 1x, there is
    something to engineer. If they are both so slow that nobody would run this
    configuration twice, the honest answer is documentation, not a mechanism.

  3. Only then, pick the shape. Two survive the src/memory.c objection,
    for different reasons:

    • Trim the multiplier, not the ceiling. The resolver already steps
      k = 3, 2, 1, floor; under pressure it would start lower. Still an
      instantaneous sample, but it bounds only the optional cache above the
      floor, so a stale reading costs throughput rather than a kill, and the
      floor refusal keeps meaning what it means. The determinism objection
      survives in weakened form and has to be argued, not waved past.
    • A reservation ledger. Each waste_open publishes its resolved budget
      and pid; a later open subtracts live reservations (kill(pid, 0) prunes
      the dead) from the cap. This is the one candidate that is capacity and
      not pressure — the same kind of number as the cgroup limit, fixed for the
      life of the process, which is exactly the criterion src/memory.c uses
      to admit an input. It also covers the different-container row that no
      container lock can. It costs a writable state directory, a file format,
      cross-user visibility, and it is advisory like everything else in this
      class.
    • Do nothing, and say so. The null hypothesis stays live until gate 1
      kills it. The engine already prints what it chose; a host that runs two
      of them can read both lines.

Non-goals

  • Any change to explicit --budget.
  • Anything that refuses an open that would otherwise have succeeded. The
    budget resolver's one refusal is budget < floor_bytes, and this must not
    add a second.
  • Windows and macOS behavior, unless gate 3 lands on the ledger, which is
    portable.

Done when

docs/LEARNED.md has a dated entry with the gate-1 numbers, and either a
mechanism behind them or the reason there is none — the second outcome being
as good as the first, which is what that file is for.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions