keepfloor

llama.cpp offloads sampling to a ggml graph as an experimental feature. Its backend implementations of top_p and min_p build the graph without ever encoding min_keep, so when a caller sets min_keep >= 2 and enables backend sampling, the offloaded path returns a strictly smaller set of candidate tokens than the CPU path does from the same logits. The min_keep floor is silently dropped.

The bug

Every offloadable sampler ships two implementations that must select the same survivor set from the same logits: a CPU _apply and a ggml-graph _backend_apply driven when sampling is offloaded through the public llama_set_sampler (marked [EXPERIMENTAL]).

The CPU paths honor the min_keep floor (src/llama-sampler.cpp, build 9760, commit 6ee0f65793):

// top_p CPU (L1384):  stop only once enough mass AND at least min_keep tokens
if (cum_sum >= p && i + 1 >= ctx->min_keep) { ... break; }
// min_p CPU (L1587):  keep while above threshold OR under the floor
if (logit >= min_logit || i < ctx->min_keep) { ... }

The backend graphs do not. llama_sampler_top_p_backend_apply (L1427-1498) and llama_sampler_min_p_backend_apply (L1618-1655) build argsort / softmax / cumsum / ggml_step / set_rows / log / add, and never reference ctx->min_keep. The offload gates do not compensate either: llama_sampler_backend_support checks only whether the backend supports the ops, and can_offload in src/llama-context.cpp checks only that the iface exposes backend_init / backend_apply. Nothing keeps a min_keep >= 2 sampler off the backend, and once there the floor is gone.

This is a backend-graph omission, the same class as a clone that copies one field but not its coupled twin.

How it is shown

The audit links the real samplers and the real ggml backend and compares the two survivor sets.

• CPU set: the real llama_sampler_apply, read from the truncated cur_p.
• Backend set: the real iface->backend_apply driven on a ggml CPU backend, mirroring the shipped llama_sampler_backend_support graph driver (ggml_init, llama_sampler_data, backend_apply, build_forward_expand, gallocr alloc, backend_graph_compute). A dropped token gets +log(0) = -INF, so a token survives iff its output logit is finite.
• Grid: sampler in {top_p, min_p}, three distinct-logit profiles, p in {0.5, 0.7, 0.9, 0.95}, min_keep in {0, 1, 2, 3, 5, 8}.

cells 144
cells where the backend drops the min_keep floor 78   (top_p 31, min_p 47)
cpu law |cpu| = max(threshold_count, min_keep) holds on every cell
the dropped tokens are exactly the lowest-logit min_keep floor on every bug cell
backend set is a subset of the cpu set on every cell

The closed-form law |cpu| = max(threshold_count, min(min_keep, n)) versus |backend| = threshold_count holds on all 144 cells, so the backend set is the CPU set minus exactly max(0, min(min_keep, n) - threshold_count) floor tokens. Example (repro.cpp):

top_p p = 0.50, min_keep = 3, 12 distinct logits:
  CPU     llama_sampler_apply -> keeps 3 tokens   (honors the min_keep floor)
  backend backend_apply       -> keeps 1 tokens   (min_keep dropped)

Why the audit is trustworthy

• Two real implementations. The harness links the shipped libllama and libggml and drives the real llama_sampler_apply and the real iface->backend_apply on a ggml CPU backend. No model, no vocab, no decode. Build 9760, commit 6ee0f65793, ggml 0.15.2.
• Self-validating control. Every min_keep in {0, 1} cell must show zero divergence, and does (48/48). Those cells run the identical sort / softmax / cumsum / step graph, so any floating-point or argsort artifact would surface there. Their agreement attributes all min_keep >= 2 divergence to the min_keep omission alone, not to the harness or to numerics.
• Exact structure, not just a count. On every bug cell the backend set is a subset of the CPU set and the missing tokens are exactly the lowest-logit floor (top_p output is a verified contiguous sorted prefix; min_p survivors are compared as sets directly), and divergence occurs on a cell if and only if min_keep > threshold_count.
• Both samplers. The drop is realized on top_p (31 cells) and min_p (47 cells).
• The mechanism is confirmed at the source line: the backend graphs contain no reference to min_keep, while the CPU paths gate on it.

Reachability (stated honestly)

min_keep is a public parameter of llama_sampler_init_top_p / llama_sampler_init_min_p (and the server min_keep field). Backend sampling is the public llama_set_sampler, marked [EXPERIMENTAL] and not the default (common's default min_keep is 0). The divergence needs min_keep >= 2 AND backend offload enabled: a public but non-default, experimental configuration. The two paths are contractually required to agree, so a divergence there is a genuine correctness bug rather than misuse; it is scoped as an experimental opt-in, not a default-path break. A separate > vs >= boundary difference (the backend ggml_step mask is strict-positive while CPU min_p keeps logit >= threshold) is noted structurally but is not the load-bearing claim.

Run

./run.sh

Needs the Homebrew llama.cpp at /opt/homebrew. Fully CPU, deterministic; no model, no decode; the ggml graphs are a dozen f32 elements and run in milliseconds.

Files

• keepfloor.cpp: links the real CPU sampler and the real backend graph; the survivor-set differential, the closed-form law check, the self-validating min_keep in {0,1} control, the subset and floor-structure checks, and the per-sampler breakdown.
• repro.cpp: minimal standalone reproduction.
• PREREG.md: the pre-registration.
• results/main.json: machine-checked output; results/summary.txt: the one-line summary.

Scope

Fully CPU, deterministic, exact; no model, no decode, no GPU. The audited quantity is the survivor set of each path. The finding is a backend-graph omission (min_keep never encoded). The interaction with a live offloaded sampler chain, the strict-boundary rider, and non-CPU backends are out of scope; the reachability boundary is stated above.

License

MIT.