Skip to content

v0.6.7 - Ling 3.0 tool calls no longer land in the reasoning channel

Choose a tag to compare

@Nathanw1014 Nathanw1014 released this 21 Aug 02:55
· 11 commits to master since this release

v0.6.7 - Ling 3.0 tool calls no longer land in the reasoning channel

If you run Ling 3.0 (BailingMoE3) with tools, some turns came back with no tool call at all:
finish_reason was stop, content was empty, and the raw <tool_call> block sat inside
reasoning_content as text. Reported upstream as
ggml-org/llama.cpp#27462.

Cause: the Bailing V3 template opens the think block in the generation prompt itself, so the model
is already inside <think> when it starts generating. Ling sometimes goes straight into
<tool_call> without closing </think> first. The auto parser had a single reasoning terminator,
</think>, so with no closing tag it read the rest of the turn, tool call included, as reasoning.

Fix: <tool_call> is now a second reasoning terminator for this template, and it is left in the
input rather than consumed, so the tool parser picks it up. This is what the hand written
Qwen3-Coder parser already does for the same behaviour. Reasoning is still captured, it just ends
where the tool call begins.

Nothing else changed in this release. The diff is 47 lines across the chat parser and its tests.
No ggml, no shaders, no backend code, so decode and prefill are identical to v0.6.6 by
construction.

Verified on the released tarball

Ling-3.0-tiny Q4_K_M, bundled RADV driver, Radeon 8060S (gfx1151), same model and flags on both
sides:

v0.6.6 v0.6.7
normal tool call works works
tool call with the think block left open finish_reason: stop, 0 tool calls, <tool_call> text inside reasoning_content finish_reason: tool_calls, reasoning and get_forecast in their own fields

The open think block was forced directly rather than waited for: banning the </think> token with
logit_bias makes it impossible for the model to close the block, which is the same input shape
the bug report describes. Streaming and non streaming both behave as the table says.

Also checked: parser test suite green (111 tests, 534 assertions), and the real Ling-3.0-flash
template from Hugging Face replayed through the server's own parse path against a stashed
baseline, which fails the same way v0.6.6 does.

Read before quoting

  • This is a parser fix. There is no performance claim here, and no benchmark was re-run: the
    backend binaries are built from the same sources as v0.6.6.
  • Known trade off: reasoning that literally quotes the string <tool_call> gets cut short at that
    point. The Qwen3-Coder parser has carried the same trade off for as long as it has had this
    handling.
  • Only templates carrying the Bailing V3 marker take the new terminator. Every other model parses
    exactly as it did in v0.6.6.

Changes since v0.6.6

  • chat: parse Bailing V3 tool calls inside an unclosed think block (9b9ac3e38)

Payload built from Nathanw1014/llama.cpp@9b9ac3e38 (branch strix-halo-vulkan), bundled Mesa RADV
devenv driver as in v0.6.4 through v0.6.6.