v0.12.0 — unparsed tool calls are no longer silent
A tool call the serving runtime did not parse used to reach you as an ordinary
assistant message. A model that had called its tools correctly for an entire
session could therefore look like it was narrating work it never did — no file
written, no error, no warning, just prose.
What changed
Unparsed tool calls now surface. The agent loop has always published a
ParseFailed event when a reply looks like a tool call but nothing parsed.
Nothing in any channel subscribed to it, so the one signal that explained the
failure went only to the benchmark metrics sink. The terminal and Discord
channels now render it with the retry count and the runtime flag worth
checking. Loop behavior is unchanged: it still nudges the model with the taught
format and retries up to three times, and a run is never killed over a single
unparsed turn.
Detection of runtime-native tool syntax arriving as chat text — DeepSeek's
DSML (V4-Flash) and the V3 tool_calls_begin envelope.
This is detection, not parsing. LocalHarness deliberately does not add
per-model tool-call parsers. Converting a model's native syntax into
structuredtool_callsis the serving runtime's job — llama.cpp--jinja
reads the template embedded in the GGUF, vLLM needs--tool-call-parser—
and it already works when that flag is set. What this release changes is that
the failure is named instead of silently rendered as prose.
The capability probe no longer condemns a session on one hiccup. Tool-call
mode for a whole session was decided by a single generation, and any failure —
a busy slot, a slow first token, a connection blip — silently downgraded every
later turn to XML mode. For a model whose native syntax is not <tool_call>,
that meant every tool call in the session was rendered as chat and nothing ran.
An inconclusive probe is now retried. A definitive answer still resolves on the
first attempt, so the healthy path still costs exactly one request. Exhausting
the retries still ends in XML mode with the error recorded — the retry widens
the window for a transient blip, it never assumes a capability the server did
not demonstrate.
Fixed
- #101 —
doctortold healthy multi-slot llama.cpp servers to relaunch. The
advisory fired on slot count alone, but modern llama.cpp shares one KV cache
across slots (kv_unified) and does not divide--ctx-size; it now compares
the per-request window from/propsagainst the configured budget. - #102 — an explicit
provider_typehard-failedstartwhen it disagreed
with the running server, which made setting it strictly worse than leaving it
blank. It now orders the/tokenizeprobe instead of narrowing it, so a
drifted config recovers with a warning. The hard error when neither shape
answers is preserved.
Caveats
- Detection covers the DeepSeek syntaxes observed in practice. A model family
emitting some other unparsed format will still render as prose until its
pattern is added — the warning exists because that class of failure cannot be
fully enumerated in advance. - The fix for an unparsed tool call is still a runtime flag, not a harness
setting. LocalHarness can now tell you that, but it cannot convert the syntax
for you.
Pre-1.0: interfaces and config schema may change without notice.