piecebounds

llama.cpp's public llama_token_to_piece crashes on an out-of-range token id. Because the token-to-piece cache is populated when a vocab loads, every call takes a fast path that does an unchecked cache.at(token); a negative or >= n_tokens id throws std::out_of_range, and since the function is a C API with no handler, the exception propagates across the C ABI boundary and terminates any caller that has no C++ handler. The function's own out-of-range guard is dead code in production.

The bug

After any vocab loads, llama_vocab::impl::load fills cache_token_to_piece (src/llama-vocab.cpp:2923-2929), so the fast path is always taken (build 9760, commit 6ee0f65793):

const auto & cache = cache_token_to_piece;
if (!cache.empty()) {
    const auto & result = cache.at(token);   // :3516  -- std::vector::at, unchecked by the caller
    return _try_copy(result.data(), result.size());
}
if (0 <= token && token < (int32_t) id_to_token.size()) { ... }   // :3521  -- guarded sibling, never reached

std::vector::at throws std::out_of_range when token < 0 (converted to a huge size_t) or token >= n_tokens. The public chain llama_token_to_piece (:4314) -> token_to_piece (:4058) -> .at has no exception handler, so the throw crosses the extern "C" boundary and, for a caller without a matching handler, reaches std::terminate -> SIGABRT. The guarded sibling at :3521 returns 0 for an out-of-range id, which is the intended behavior, but it only runs when the cache is empty, so it is dead once a vocab is loaded.

How it is shown

The audit links the real llama_token_to_piece and drives it on a hand-built ~200-byte synthetic SPM vocab (loaded vocab_only), with two agreeing oracles.

n_tokens 7, valid ids 0..6
out-of-range ids (-2^31, -100, -2, -1, 7, 8, 57, 2^31-1): all crash the handler-less child (SIGABRT)
in-range ids (0, 1, 3, 6): all return real bytes
crash boundary is exactly [0, n_tokens): yes
the subprocess crash and the in-process std::out_of_range catch agree on every id
token 6 returns, token 7 crashes: only the id differs

The repro.cpp output:

vocab has 7 tokens (valid ids are 0..6)
  llama_token_to_piece(token = 3):   returned a piece
  llama_token_to_piece(token = 6):   returned a piece   (last valid)
  llama_token_to_piece(token = 7):   CRASH (uncaught std::out_of_range -> terminate)
  llama_token_to_piece(token = -1):  CRASH (uncaught std::out_of_range -> terminate)

Why the audit is trustworthy

• Real subject. The harness links the shipped libllama and calls the real llama_token_to_piece on a real vocab loaded through llama_model_load_from_file. Build 9760, commit 6ee0f65793.
• Two agreeing discrete oracles. A forked child with no catch aborts (the production consequence for a handler-less caller); an in-process catch (const std::out_of_range &) fires on the same ids, proving the crash is the .at() throw and not an unrelated abort. They agree on every id, and the crash boundary is exactly [0, n_tokens).
• Armed self-test. A std::vector<int>(1).at(5) in a try/catch is caught, proving the catch detects std::out_of_range and a silently non-throwing API could not masquerade as a pass.
• Non-vacuity. Every in-range id returns real bytes, so the production cache path is actually exercised and the real code is linked, not a stub.
• Minimal one-id flip. token = 6 returns and token = 7 crashes with only the integer differing, isolating the boundary to the token argument.

Reachability (stated honestly)

llama_token_to_piece / llama_detokenize are stable public C API declared extern "C". Throwing a C++ exception across that boundary means a C consumer or a language-binding FFI caller (Python ctypes, Go cgo, Rust, node) cannot catch it, so passing an unchecked token id terminates the process. The function's own dead guarded sibling shows out-of-range was meant to be handled gracefully.

The bundled llama-server is NOT affected. Its ex_wrapper wraps every route handler in catch (const std::exception &) (tools/server/server.cpp:40-54, with the comment "make sure handler_t never throws"), so /detokenize catches the std::out_of_range and returns an error response rather than crashing. The exposure is therefore to direct C-API and binding callers that forward a token id without bounds-checking it, not to the server endpoint. The consequence is a crash for a handler-less caller, not memory corruption or remote code execution.

Run

./run.sh

Needs the Homebrew llama.cpp at /opt/homebrew. Fully CPU, deterministic; no real model, no decode; the vocab is a synthetic sub-KB GGUF built in memory, written to a temp file, loaded vocab_only, and deleted.

Files

• piecebounds.cpp: the in-memory vocab builder, the vocab load, the boundary sweep with the subprocess and in-process oracles, the armed self-test, the non-vacuity check, and the one-id-flip isolation.
• repro.cpp: minimal standalone reproduction.
• vgguf.h: the minimal GGUF vocab byte builder (no external tooling).
• PREREG.md: the pre-registration.
• results/main.json: machine-checked output; results/summary.txt: the one-line summary.

Scope

Fully CPU, deterministic; no real model, no decode, no GPU. The audited quantity is whether the public llama_token_to_piece crashes on an out-of-range token id. The finding is an unchecked cache.at(token) on the always-taken fast path, with the intended out-of-range handling stranded in a dead branch. The interaction with a full model vocab (any loaded vocab has the same populated cache and the same crash), and callers that do wrap the API in a handler, are out of scope.

License

MIT.