llama.cpp backend: tool schemas with string maxLength >= ~2000 break all tool calling (lemonade launch claude unusable out of the box)

[jflaflamme]

Summary

Any tool whose JSON schema declares a string maxLength at or above ~2000 makes every tool-calling request to a llama.cpp backend fail with HTTP 400:

parse: error parsing grammar: number of repetitions exceeds sane defaults, please reduce the number of repetitions
E srv send_error: Failed to initialize samplers: failed to parse grammar

llama.cpp compiles each tool's schema into a GBNF grammar to constrain tool-call decoding. A string maxLength: N is emitted as char{0,N}. Since ggml-org/llama.cpp#17381 added a repetition sanity cap (a DoS guard), llama.cpp now generates a grammar that its own parser then rejects. See also ggml-org/llama.cpp#17473 — closed, but the schema→GBNF converter path still trips on the currently pinned builds.

The failure is model-independent: it's purely a function of the tool schema.

Impact: lemonade launch claude is unusable out of the box

Claude Code's built-in Workflow tool declares its script parameter with maxLength: 524288, which the converter expands to:

tool-Workflow-schema-script ::= "\"" char{0,524288} "\""

So every tool call 400s and the agent can't do anything. The only workaround is --agent-args "--tools <explicit list omitting Workflow>", which is undiscoverable — nothing in the error points at a tool schema, and the Claude Code integration docs don't mention it.

Note --disallowedTools does not work: it only denies permission, so the schema is still sent and still breaks the grammar. --tools (which selects from the built-in set) is required.

This isn't Claude-specific — llama.cpp#17473 was originally reported via OpenWebUI native tool calls, and this repo has adjacent tool-parse reports (#1562, #1011).

Reproduction

Any llama.cpp model. The only difference between the two calls is the presence of maxLength.

# 400 — Failed to initialize samplers: failed to parse grammar
curl -s localhost:8000/api/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "<any-llamacpp-model>",
  "messages": [{"role":"user","content":"hi"}],
  "tools": [{"type":"function","function":{"name":"T","description":"t","parameters":{
    "type":"object","properties":{"s":{"type":"string","maxLength":524288}},"required":["s"]}}}]
}'

# 200 — identical, maxLength removed
curl -s localhost:8000/api/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "<any-llamacpp-model>",
  "messages": [{"role":"user","content":"hi"}],
  "tools": [{"type":"function","function":{"name":"T","description":"t","parameters":{
    "type":"object","properties":{"s":{"type":"string"}},"required":["s"]}}}]
}'

Measured threshold — gpt-oss-120b, llamacpp vulkan b9747, Ryzen AI MAX (Strix Halo, gfx1151), Lemonade 10.9.0:

string maxLength	result
1000	200 OK
2000	400 — failed to parse grammar
4096 / 65536 / 524288	400

End-to-end: lemonade launch claude --agent-args "--tools Bash,Read,Write,Edit,Glob,Grep,WebFetch,Skill,Agent,TaskCreate" works; adding Workflow to that same list reproduces the 400.

Proposed fix

Sanitize tool schemas on the llama.cpp forwarding path, as a request transform alongside the existing JsonUtils::with_* helpers:

// src/cpp/include/lemon/utils/json_utils.h
static json with_grammar_safe_tool_schemas(const json& request);

It would walk tools[].function.parameters (recursively through properties / items / $defs) and remove any maxLength / maxItems at or above a conservative threshold (1000 measured safe).

Apply it in:

• LlamaCppServer::chat_completion — compose with the existing transform, exactly as vllm_server.cpp already composes two:

  return forward_request("/v1/chat/completions",
                         JsonUtils::with_grammar_safe_tool_schemas(
                             JsonUtils::with_legacy_max_tokens_alias(request)));

• LlamaCppServer::responses — same, if it accepts tools.
• Router::chat_completion_stream — ⚠️ this is the one that's easy to miss. It forwards the raw string body, so it bypasses the transform above entirely. Claude Code streams, so patching only chat_completion() would pass a curl test and still leave the agent broken. It already parses the body into request_json, and identity.recipe is in scope at the forward_streaming_request call site, so the sanitized JSON can be re-dumped for llama.cpp-recipe backends.

Why remove the bound rather than clamp it

Clamping maxLength down to the cap would be a correctness bug: it would forbid the model from ever emitting a legitimately long argument (a Claude Code Workflow.script really can be hundreds of KB). Dropping the constraint yields char* — unconstrained — which is the correct semantics. maxLength is consumed here only to build the decoding grammar; Lemonade isn't acting as a schema validator, and the field is still forwarded to the model in the tool description.

Scope

llama.cpp backends only — cloud / vLLM / FLM are unaffected and shouldn't be touched.

Precedent

This matches the existing llama.cpp-workaround pattern in llamacpp_server.cpp (the HOME-unset segfault guard, ~L535), which carries a "delete once the upstream fix lands in backend_versions.json" comment linking the upstream PR. The same comment style would apply here, referencing ggml-org/llama.cpp#17473 / #17381, so the shim can be removed once the pinned llama.cpp clamps the bound in its converter.

Alternative considered and rejected

Hardcoding --tools <list> into the lemonade launch claude preset. It hardcodes another project's tool names (which drift), silently strips capability from the user, only fixes Claude Code while OpenWebUI/AnythingLLM hit the same class of failure, and does nothing for direct API consumers. The schema sanitizer fixes all of them in one place.

I'm happy to put up a PR if the approach and threshold look right — I didn't want to guess at your $defs/recursion conventions.

Environment

• Lemonade 10.9.0, server port 13305
• Backends: llamacpp vulkan b9747 and rocm b9752 (both affected — it's schema-driven, not backend-driven)
• Model: gpt-oss-120b-GGUF-UD-Q6_K_XL (reproduces on any llama.cpp model)
• AMD Ryzen AI MAX / Strix Halo (gfx1151), 128 GB unified, Ubuntu 26.04, kernel 7.0
• Claude Code (current), launched via lemonade launch claude