Eval bug: tool-call grammar build fails (HTTP 400 "failed to parse grammar") above ~58 tools — cumulative schema-size ceiling, distinct from #25746/#24658

[headbouyJB]

Name and Version

version: b454 (571d0d540), HIP/ROCm build (gfx1151 / Strix Halo), llama-server with --jinja. Behaviour also confirmed present on 2f18fe1 (Jun 26). (The failure is in json-schema→GBNF grammar construction, which is backend-independent — reproducible on any build.)

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server (--jinja tool-call grammar / json-schema-to-grammar)

Problem description & steps to reproduce

When the request carries a large-but-realistic tool set, llama-server rejects it at grammar-build time:

HTTP 400 {"error":{"code":400,"message":"Failed to initialize samplers: failed to parse grammar"}}

The rejection arrives in ~0.13 s with no prefill and no GPU activity — it dies while compiling the combined tool-call grammar, before any inference. Single small tool sets work normally.

The trigger is cumulative schema size, not tool count and not any individual tool. Demonstrated by holding tool count constant and varying only total schema volume:

tools	tools-array JSON size	result
58 (set A)	97,102 B	HTTP 200
58 (set B — same count, larger-schema tools)	100,026 B	HTTP 400

Same count, different composition → the larger-volume set fails. So it is not tool count, and not a single culprit schema (the 50-tool base is identical in both sets; a fixed bad schema would fail both).

Boundary scan (base of 50 real tools + N additional real MCP tools), current build:

total tools	tools-array size	result
58	97,102 B	200
59	97,935 B	400
60	98,951 B	400
65	103,348 B	400
89	121,115 B	400

The 200→400 crossover sits at ~97–98 KB of tools-array JSON (proxy for compiled-grammar size) for this tool mix. All failures return the identical error in ~0.1 s.

Real-world impact: the agent framework driving this (OpenClaw) treats the 400 as a provider failure and silently falls back to a cloud model — local inference stops with no user-visible error. The failure appeared when an installed MCP server's tools pushed the materialized set past the threshold.

Not explained by existing issues

• common : simplify autoparser tagged parser rules #21216 (MAX_REPETITION_THRESHOLD, top-level): merged April 2026; present in this build (b454). Reproduces regardless.
• Misc. bug: json-schema-to-grammar — nested string maxLength >= 2000 emits un-parseable GBNF (tool-call grammar, b10034) #25746 (nested string maxLength ≥ 2000): static scan of all 96 tool schemas in the failing payload finds zero strings with maxLength ≥ 2000 and zero pattern fields at any depth. That trigger is absent here; and the cumulative-size behaviour above is inconsistent with a single-schema trigger.
• Eval bug: peg-native: tool-call grammar generator emits GBNF that fails its own parser (Kimi K2.7-Code, 22-tool fan-out) #24658 (peg-native multi-call): different chat format (Qwen3.5 jinja, not peg-native) and different failure point (request-time grammar build vs post-generation parse).
• Not tool count: 120 trivial synthetic tools build fine on the same binary; the variable is cumulative schema volume.
• Template-dependent: the same tool payload that builds under the Qwen3.5 template fails under a Qwen3 template at a lower count — consistent with a total-compiled-grammar-size limit, since the template contributes to the grammar.

Suspected mechanism

Total compiled GBNF size / rule-count exceeding an internal parser limit as tool schemas accumulate — a volume ceiling in json-schema-to-grammar + the GBNF parser, distinct from the shape triggers already tracked. Secondary observation: in default single-slot config this error is fatal and exits the server; with --parallel 2 the server survives and returns the 400.

Reproduction / offer

Reproducible on gfx1151 (ROCm) with Qwen3.5-122B-A10B and a real 96-tool MCP payload; the grammar-build path is backend-independent so it should reproduce anywhere with a comparably-sized tool set. I have a probe harness (adds real tool schemas to a base set; failures cost ~0.1 s / no prefill) and can share the payloads, and I'm happy to test candidate fixes on this hardware.

(Investigation and write-up done with AI assistance; all HTTP results from live runs on the machine described, reproducible on request.)

[headbouyJB]

Follow-up — ruling out a single pathological tool more rigorously (rather than just a culprit in the fixed base). Two additional data points on the same build:

Two disjoint failing sets. Starting from the same 50-tool base, I added two non-overlapping groups of real MCP tools:

set	added tools	total tools	tools-array size	result
P	9 largest-schema tools	59	101,041 B	400
Q	16 smallest-schema tools	66	98,006 B	400

The added tool sets are disjoint (∩ = ∅), yet both fail — so failure cannot depend on any one specific tool being present. Note also Q has more tools (66) than P (59) but fewer bytes, and still fails: it tracks cumulative schema size, not count.

No individually-pathological tool. Adding just the single largest-schema tool to the base (51 tools, 90,695 B) returns HTTP 200 normally. So no single tool trips the grammar on its own.

Together with the equal-count/different-size pair in the original report, this points to a cumulative compiled-grammar-size limit rather than any per-tool schema shape. Failing probes remain ~0.1 s (no prefill), so this is cheap to bisect further if a maintainer wants a specific threshold characterized.

[headbouyJB]

Correction / retraction. After further controlled testing I mis-diagnosed this. The failures are not a cumulative schema-size ceiling — that was a spurious correlation. Closing this as a duplicate of the nested-pattern GBNF bug (#22314, and the nested-pattern case discussed in #25746).

Real root cause, isolated by ablation on the same build (b454-571d0d5):

One of the tools I was adding at the "boundary" (batch-delete-messages) carries a string pattern nested inside array items:

"ids": { "type": "array",
         "items": { "type": "string", "pattern": "^(\\d+|imap:[A-Za-z0-9_-]+)$" },
         "minItems": 1, "maxItems": 100 }

Adding just that one tool to an otherwise-working 50-tool set (→ 51 tools, 90 KB — well under any size threshold) → HTTP 400 Failed to initialize samplers: failed to parse grammar. Changing only that nested pattern:

variant (single tool added to the same 50-tool base)	result
nested pattern present (original)	400
nested pattern removed (all else identical)	200
same pattern moved to a top-level property	200
maxItems removed / changed, pattern kept	400 (no effect)

So the trigger is a nested string pattern; the identical regex at top level is fine, and maxItems is irrelevant. My earlier "cumulative size" and "no single tool trips it" claims were wrong — a single nested-pattern tool trips it, and the apparent size/count correlation was just that most of the tools I happened to be adding (the batch-* / message-id tools) carry this same nested pattern. Apologies for the noise. Clean repro contributed to #25746 where the nested-constraint handling is being worked on.