penaltyledger

Cloning llama.cpp's penalties sampler silently drops its penalty state, so a cloned sampler stops penalizing repeats and, after a few more tokens, starts boosting them. The sampler's normal accept-time bookkeeping is otherwise exact.

The penalties sampler (llama_sampler_penalties, repetition/frequency/presence penalties) keeps an O(1) incremental histogram token_count alongside a ring buffer prev of the last penalty_last_n accepted tokens. They must agree at all times; the maintainers wrote a sanity assertion for exactly this and left it in the source behind #if 0 (src/llama-sampler.cpp L2658-2665). This repo re-arms that assertion as a live differential oracle against the real shipped sampler, and finds a clone bug.

Result

1. A real bug in llama_sampler_penalties_clone. The clone copies the ring buffer but not the histogram:

   result_ctx->prev = ctx->prev;   // copies the window ...
   // ... but token_count is never copied; it stays empty from init

   So a cloned penalties sampler has a populated window (prev) and an empty count map (token_count), an inconsistent state the sampler's own #if 0 assertion forbids. The consequences, demonstrated on a fresh context (repro.cpp):

   original:   penalty(5)=2  penalty(6)=1        (window [5,5,6]: correct)
   clone:      penalty(5)=0  penalty(6)=0        (clone forgot the penalty state)
   clone+7+8:  penalty(5)=-1 penalty(6)=0        (5 gets a NEGATIVE penalty = boosted; 6 unpenalized)
   

   Immediately after cloning, the clone applies no penalty to the tokens in its window (it silently forgets the accumulated state). Worse, once the clone accepts enough new tokens to evict a pre-clone token, the eviction decrements a count that was never set, driving it negative; apply then subtracts a negative penalty, so a repeated token is boosted instead of suppressed, the exact opposite of a repetition penalty. On a build with assertions enabled the same bug surfaces differently: apply has assert(count > 0 && count <= penalty_last_n), which the negative count trips, so the process aborts instead of boosting. The shipped Homebrew release is NDEBUG (the assertion is compiled out, confirmed because repro prints -1 rather than aborting), so the release consequence is the silent boost. Either way it is a real defect. It is reachable through the public llama_sampler_clone, which clones a sampler chain for parallel and speculative decoding and for saving or restoring sampler state. The fix is one line: the clone must also copy token_count (or rebuild it from prev).

2. The accept-time ledger is exact. Away from clone, the O(1) token_count ledger equals the histogram of the window on every reachable state: over 3,010,612 exhaustively enumerated accept-sequences (alphabets of 2-5 token ids, penalty_last_n in {1,2,3,4,8}, lengths chosen to fill and evict the ring many times, including the full-ring self-collision where the incoming token equals the one being evicted), the ledger and the from-scratch window histogram agree with zero mismatches. reset is also correct (it clears both structures). So the maintainers' disabled assertion would hold for accept and reset; only clone breaks it.

Why the audit is trustworthy

• Real subject. The harness links the shipped libllama and drives the real llama_sampler_init_penalties / accept / apply / clone / reset. No model, no vocab, no decode. Build 9760, commit 6ee0f65793.
• The private ledger is read exactly through the public API. With penalty_repeat = 1 (a no-op), present = 0, penalty_freq = F, and a candidate logit L, apply computes logit' = L - count * F, so count is recovered as (L - logit') / F. With L and F powers of two and count bounded, this is bit-exact in fp32, and it never touches the private field. The math oracle is validated in isolation first, so a math discrepancy cannot masquerade as an accounting one.
• Two genuinely different algorithms. The sampler's O(1) incremental delta versus a from-scratch O(n) window histogram, compared by exact integer equality. A finite, exhaustively enumerated domain makes the accept-path result a proof over that domain.
• Bidirectional positive control. A token in the window reads a nonzero count; after penalty_last_n distinct other accepts evict it, it reads exactly zero; and a deliberately off-by-one shadow is flagged, proving the comparator fires in both directions.
• The bug reproduces standalone. repro.cpp reproduces it on a fresh context with no audit machinery, and the mechanism is confirmed at the source line (clone assigns prev but not token_count).

Run

./run.sh

Needs the Homebrew llama.cpp at /opt/homebrew. Fully CPU, deterministic; no model, no decode, no GPU; runs in a few seconds.

Files

• penaltyledger.cpp — links the real sampler; the public-API count readout, the from-scratch window histogram, the exhaustive accept-sequence sweep, the reset and clone sub-audits, and the bidirectional control.
• repro.cpp — minimal standalone reproduction of the clone bug and the boost consequence.
• PREREG.md — the pre-registration.
• results/main.json — machine-checked output.

Scope

Fully CPU, deterministic, exact; no model, no decode, no GPU. The audited quantity is the integer per-token token_count, which fully determines the penalty. The certificate covers the accept and reset paths over the enumerated domain; the clone bug is a state-copy omission independent of penalty parameters. Other samplers and the interaction with a full sampler chain are out of scope.

License

MIT.