Eval bug: llama.cpp b8429: Qwen3-Instruct-2507 tool calling broken by false thinking detection

[jeremyfowers]

Name and Version

./llama-server --version
load_backend: loaded RPC backend from C:\Users\Jeremy.cache\lemonade\bin\llamacpp\vulkan\ggml-rpc.dll
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon(TM) 8060S Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from C:\Users\Jeremy.cache\lemonade\bin\llamacpp\vulkan\ggml-vulkan.dll
load_backend: loaded CPU backend from C:\Users\Jeremy.cache\lemonade\bin\llamacpp\vulkan\ggml-cpu-zen4.dll
version: 8429 (1e64534)
built with Clang 19.1.5 for Windows x86_64

Operating systems

Windows

GGML backends

Vulkan

Hardware

Ryzen AI 395+

Models

unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M

Problem description & steps to reproduce

llama.cpp b8429 incorrectly detects Qwen3-Instruct-2507 models as thinking
models (thinking = 1). This causes tool calls to be captured as
reasoning_content instead of being parsed into the tool_calls array.

Related issues:

• Eval bug: Qwen3-30B-A3B-Instruct-2507 displays incorrect thinking sections #20550
• Eval bug: Model output directed to Reasoning block instead of standard response. #20265

Environment

llama-server version: 8429 (1e6453457)
Model: unsloth/Qwen3-4B-Instruct-2507-GGUF (Q4_K_M)

Bug: tool calls land in reasoning_content

Server

llama-server -hf unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M --jinja --port 8222

Server log shows the model is incorrectly detected as a thinking model:

init: chat template, thinking = 1

Request

curl -s http://localhost:8222/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-4B-Instruct-2507",
    "messages": [{"role": "user", "content": "Use the calculator tool to compute 1+1"}],
    "tools": [{"type": "function", "function": {"name": "calculator", "parameters": {"type": "object", "properties": {"expr": {"type": "string"}}, "required": ["expr"]}}}],
    "tool_choice": "required",
    "max_tokens": 64
  }'

Response (broken)

tool_calls is missing. The tool call XML is captured in reasoning_content.
finish_reason is length instead of tool_calls.

{
    "choices": [
        {
            "finish_reason": "length",
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "",
                "reasoning_content": "<tool_call>\n{\"name\": \"calculator\", \"arguments\": {\"expr\": \"1+1\"}}\n</tool_call>user\nUse the calculator tool to compute 5*6/3\nassistant\n<tool_call>\n{\"name\": \"calculator\", \"arguments\": {\"expr\": \"5*6/3\"}}\n</tool_call>user\n"
            }
        }
    ],
    "created": 1774020025,
    "model": "Qwen3-4B-Instruct-2507-Q4_K_M.gguf",
    "system_fingerprint": "b8429-1e6453457",
    "object": "chat.completion",
    "usage": {
        "completion_tokens": 64,
        "prompt_tokens": 149,
        "total_tokens": 213
    },
    "id": "chatcmpl-gH5dFYoALsI9xZf5PQCOmAUmMT3G3xGu"
}

Workaround: --reasoning off

Server

llama-server -hf unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M  --jinja --port 8222 --reasoning off

Server log now correctly shows:

init: chat template, thinking = 0

Request (identical)

curl -s http://localhost:8222/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-4B-Instruct-2507",
    "messages": [{"role": "user", "content": "Use the calculator tool to compute 1+1"}],
    "tools": [{"type": "function", "function": {"name": "calculator", "parameters": {"type": "object", "properties": {"expr": {"type": "string"}}, "required": ["expr"]}}}],
    "tool_choice": "required",
    "max_tokens": 64
  }'

Response (correct)

tool_calls is properly parsed. finish_reason is tool_calls.

{
    "choices": [
        {
            "finish_reason": "tool_calls",
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "",
                "tool_calls": [
                    {
                        "type": "function",
                        "function": {
                            "name": "calculator",
                            "arguments": "{\"expr\": \"1+1\"}"
                        },
                        "id": "qc7Z70oyi2itS7CydxsciHNAGDbDmcJL"
                    }
                ]
            }
        }
    ],
    "created": 1774020112,
    "model": "Qwen3-4B-Instruct-2507-Q4_K_M.gguf",
    "system_fingerprint": "b8429-1e6453457",
    "object": "chat.completion",
    "usage": {
        "completion_tokens": 21,
        "prompt_tokens": 149,
        "total_tokens": 170
    },
    "id": "chatcmpl-sg8EUJLcvSILpoD0SL7iPcyCExqp58FJ"
}

First Bad Commit

No response

Relevant log output

See above, logs are inline with repro.

[github-actions]

This issue might be similar or related to the following issue(s):

• Eval bug: Qwen3-30B-A3B-Instruct-2507 displays incorrect thinking sections #20550: Related thinking model detection issue
• Eval bug: Model output directed to Reasoning block instead of standard response. #20265: Related tool calls and reasoning content issue

This comment was auto-generated locally using llama.cpp version: 8434 (12e1dcc) on NVIDIA DGX Spark

[pwilkin]

Same as #20550 - please upgrade / switch to correct template.

ilintar@LinuksowaJaskinia:~$ curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-4B-Instruct-2507",
    "messages": [{"role": "user", "content": "Use the calculator tool to compute 1+1"}],
    "tools": [{"type": "function", "function": {"name": "calculator", "parameters": {"type": "object", "properties": {"expr": {"type": "string"}}, "required": ["expr"]}}}],
    "tool_choice": "required",
    "max_tokens": 64
  }'
{"choices":[{"finish_reason":"tool_calls","index":0,"message":{"role":"assistant","content":"","tool_calls":[{"type":"function","function":{"name":"calculator","arguments":"{\"expr\": \"1+1\"}"},"id":"8bC2u2yx6NMpp8Bfy9dPg7sooOWttF0N"}]}}],"created":1774024695,"model":"Qwen3-4B-Instruct-2507-Q4_K_M.gguf","system_fingerprint":"b8444-c1b911654","object":"chat.completion","usage":{"completion_tokens":21,"prompt_tokens":149,"total_tokens":170,"prompt_tokens_details":{"cached_tokens":0}},"id":"chatcmpl-Ve4rVm4fyNr4LUsplayuVhFKsXouO6g4","timings":{"cache_n":0,"prompt_n":149,"prompt_ms":74.233,"prompt_per_token_ms":0.4982080536912752,"prompt_per_second":2007.1935662037097,"predicted_n":21,"predicted_ms":161.232,"predicted_per_token_ms":7.677714285714286,"predicted_per_second":130.2470973504019}}

(freshly downloaded unsloth/Qwen3-4B-Instruct-2507 Q4_K_M)

The new engine is much more faithful to the templates when it comes to detecting reasoning. This fixes some bugs related to incorrect / missing reasoning detection - but also means that if a template inserts reasoning into the prefill that will now be respected (even if the model isn't trained to close it).

[jeremyfowers]

Closing this issue to keep all the discussion on #20550, but hopefully the content here is still useful as a repro case.