Feature Request: Qwen3-Coder Tool Call Parser

[bold84]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Here is a short explanation of the differences between the chat templates for Qwen3-Coder and Qwen3-Instruct, with clear examples.

The primary difference is how they format tool calls:

• Qwen3-Coder uses a completely custom XML format for both defining and calling tools.
• Qwen3-Instruct uses standard JSON for tool definitions and expects JSON output from the model, wrapped in a simple <tool_call> tag.

---

1. Qwen3-Coder Template (Custom XML Format)

This template instructs the model to generate tool calls using unique XML tags for the function name and each parameter [2]. This format is not standard and requires a special parser to convert the XML into a usable JSON object [3].

Example Tool Call Output:

<tool_call>
  <function=search_products>
    <parameter=query>
      waterproof running shoes
    </parameter>
    <parameter=sort_by>
      price_low_to_high
    </parameter>
  </function>
</tool_call>

Key Takeaway: The entire call is structured with custom XML tags like <function=...> and <parameter=...> [2]. This is why it fails in systems that expect JSON, causing parsing errors [1].

---

2. Qwen3-Instruct Template (JSON in XML Format)

This template provides the tool definitions to the model as standard JSON. It then instructs the model to respond with a JSON object containing the function name and arguments, all inside a simple <tool_call> wrapper [4].

Example Tool Call Output:

<tool_call>
{"name": "search_products", "arguments": {"query": "waterproof running shoes", "sort_by": "price_low_to_high"}}
</tool_call>

Motivation

Well, don't we want native tool call support for coder models?

Possible Implementation

https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/main/qwen3coder_tool_parser.py

They have a custom parser that converts the model output to JSON.

[marceldev89]

Related issue #14915

[cinu]

I've made some quick-and-dirty fix here: https://github.com/cinu/llama.cpp/tree/qwen3-coder

However, there could still be some issues.

[bold84]

I'm implementing a proper solution...

[bold84]

#15019

Someone here with enough power to test the 480B version? Because it seems to me that the 30B version isn't all to capable to to do tool calling. More complex tool calls fail.
But I might overlook something. I have tried different 8bit quants, same result.

So, please feel free to test the changes in the PR. I will be back tomorrow.

Update: It actually works fine, but the 30B gguf 4 and 8 bit quantized models tend to output many 0000000000 for numerical values. Sampling issue.
I tested the FP8 version with vllm and it worked flawlessly. So, it's probably an issue with the quantization and improvement should be possible.

[pwilkin]

If you're still interested, I got the fixed template working with Qwen3-Coder (it seems the guys at Unsloth did a bit of overkill).

{# SPDX-License-Identifier: Apache-2.0 #}
{# Copyright 2025-present the Unsloth team. All rights reserved. #}

{% macro render_item_list(item_list, tag_name='required') %}
    {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
        {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
            {{- '[' }}
                {%- for item in item_list -%}
                    {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
                    {%- if item is string -%}
                        {{ "`" ~ item ~ "`" }}
                    {%- else -%}
                        {{ item }}
                    {%- endif -%}
                {%- endfor -%}
            {{- ']' }}
        {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
    {%- endif %}
{% endmacro %}

{%- if messages[0]["role"] == "system" %}
    {%- set system_message = messages[0]["content"] %}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {%- set loop_messages = messages %}
{%- endif %}

{%- if not tools is defined %}
    {%- set tools = [] %}
{%- endif %}

{%- if system_message is defined %}
    {{- "<|im_start|>system\n" + system_message }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
    {%- endif %}
{%- endif %}
{%- if tools is iterable and tools | length > 0 %}
    {{- "\n\nYou have access to the following functions:\n\n" }}
    {{- "<tools>" }}
    {%- for tool in tools %}
        {%- if tool.function is defined %}
            {%- set tool = tool.function %}
        {%- endif %}
        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
        {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
        {{- '\n<parameters>' }}
        {%- for param_name, param_fields in tool.parameters.properties|items %}
            {{- '\n<parameter>' }}
            {{- '\n<name>' ~ param_name ~ '</name>' }}
            {%- if param_fields.type is defined %}
                {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
            {%- endif %}
            {%- if param_fields.description is defined %}
                {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
            {%- endif %}
            {{- render_item_list(param_fields.enum, 'enum') }}
            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
            {%- for json_key in param_fields %}
                {%- if json_key not in handled_keys %}
		    {%- set normed_json_key = json_key %}
                    {%- if param_fields[json_key] is mapping %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson | safe) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- else %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- endif %}
                {%- endif %}
            {%- endfor %}
            {{- render_item_list(param_fields.required, 'required') }}
            {{- '\n</parameter>' }}
        {%- endfor %}
        {{- render_item_list(tool.parameters.required, 'required') }}
        {{- '\n</parameters>' }}
        {%- if tool.return is defined %}
            {%- if tool.return is mapping %}
                {{- '\n<return>' ~ (tool.return | tojson | safe) ~ '</return>' }}
            {%- else %}
                {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
            {%- endif %}
        {%- endif %}
        {{- '\n</function>' }}
    {%- endfor %}
    {{- "\n</tools>" }}
    {{- '\n\nIf you choose to call a function ONLY reply in the following format:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nvalue_2\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Use only the above XML format with correct tag nesting.\n- `function=` tag must appear exactly like shown.\n- Each parameter must have opening and closing tags.\n- Multiline values are supported inside <parameter>...</parameter>.\n- No trailing output or suffix allowed after </tool_call>.\n</IMPORTANT>' }}
{%- endif %}
{%- if system_message is defined or (tools is iterable and tools | length > 0) %}
    {{- '<|im_end|>\n' }}
{%- endif %}

{%- for message in loop_messages %}
    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
            {{- '\n' + message.content | trim + '\n' }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
            {%- if tool_call.arguments is defined %}
                {%- for args_name, args_value in tool_call.arguments|items %}
                    {{- '<parameter=' + args_name + '>\n' }}
                    {%- set args_value = args_value if args_value is string else args_value | string %}
                    {{- args_value }}
                    {{- '\n</parameter>\n' }}
                {%- endfor %}
            {%- endif %}
            {{- '</function>\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role in ["user", "system", "assistant"] %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.previtem and loop.previtem.role != "tool" %}
            {{- '<|im_start|>user\n' }}
        {%- endif %}
        {{- '<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>\n' }}
        {%- if not loop.last and loop.nextitem.role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- elif loop.last %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- else %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- endif %}
{%- endfor %}

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}

[marceldev89]

If you're still interested, I got the fixed template working with Qwen3-Coder (it seems the guys at Unsloth did a bit of overkill).

This crashes the server again with and without #15019.

terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid diff: 'I'll create a snake game in C using ncurses. Let me first check what files we have and then implement the game.
<tool_call>
<function=list' not found at start of 'I'll create a snake game in C using ncurses. Let me first check what files we have and then implement the game.
<tool_call>

'

[pwilkin]

That's weird, I literally tested it just now without the PR and seems to be working fine for me on newest build.

[pwilkin]

My config:

LLAMA_SET_ROWS=1 build/bin/llama-server --port 2345 -m /devel/alt/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf -ot "exps=CPU" -ctk q4_0 -ctv q4_0 -fa -c 200000 --jinja --temp 0.7 --top-k 20 --top-p 0.8 --min-p 0 -ngl 99 --repeat-penalty 1.05 -nr --timeout 3600 --mlock --chat-template-file qwen.jinja

qwen.jinja is what I copy-pasted above

[shekohex]

@pwilkin
Just used it without #15019 to test it out, and got a crash (latest llama-server on 9c35706):

Grammar triggered on regex: '<function=bash>
'
srv  update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash>

Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash>

Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <<<
>>>
/root/llama.cpp/build/bin/libggml-base.so(+0x14ee8)[0x77bd3c1e8ee8]
/root/llama.cpp/build/bin/libggml-base.so(ggml_print_backtrace+0x1e4)[0x77bd3c1e92b4]
/root/llama.cpp/build/bin/libggml-base.so(+0x24e29)[0x77bd3c1f8e29]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa8e1a)[0x77bd72aa8e1a]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa8e85)[0x77bd72aa8e85]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa90d8)[0x77bd72aa90d8]
/root/llama.cpp/build/bin/llama-server(+0x3b50e)[0x57172b97550e]
/root/llama.cpp/build/bin/llama-server(+0x16ec1e)[0x57172baa8c1e]
/root/llama.cpp/build/bin/llama-server(+0xd757e)[0x57172ba1157e]
/root/llama.cpp/build/bin/llama-server(+0xd7c68)[0x57172ba11c68]
/root/llama.cpp/build/bin/llama-server(+0xd8ac4)[0x57172ba12ac4]
/root/llama.cpp/build/bin/llama-server(+0xec0f5)[0x57172ba260f5]
/root/llama.cpp/build/bin/llama-server(+0xb185a)[0x57172b9eb85a]
/root/llama.cpp/build/bin/llama-server(+0x6c16b)[0x57172b9a616b]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x77bd72c4624a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x77bd72c46305]
/root/llama.cpp/build/bin/llama-server(+0x7b221)[0x57172b9b5221]
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid diff: 'I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash' not found at start of 'I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>

'
Aborted

Here is how I ran it:

LLAMA_SET_ROWS=1 /root/llama.cpp/build/bin/llama-server --host 0.0.0.0 --port 8080   --model /root/models/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf   --chat-template-file /root/models/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/chat-template.jinja   --jinja   --mlock   --flash-attn   --kv-unified   --swa-full   --verbose   --verbose-prompt   --cache-type-k q4_1   --cache-type-v q4_1   --threads -1   --n-gpu-layers 99   --ctx-size 131072   --batch-size 4096   --ubatch-size 4096   --temp 0.7   --min-p 0.0   --top-p 0.7   --top-k 20   --repeat-penalty 1.6   --presence_penalty 0.3   --keep 32768   --cache-reuse 4096   -ot "\.(8|9|1[0-9]|2[0-9]|3[0-9])\.ffn_(up|gate|down)_exps.=CPU" --alias qwen3-coder

Full Logs: Click me

``` 2 Backend API server : Using Honno web framework 3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>||||||)?(\s*{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 96, n_remaining = 8096, next token: 151657 '<tool_call>'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 118
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 119, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7962, n_cache_tokens = 7962, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"\n"}}],"created":1754093086,"id":"chatcmpl-WwOVdpVGEkfJB5DuAkNO9Q8P3hbDPBpO","model":"qwen3-coder","system_fingerprint":"b

Grammar still awaiting trigger after token 198 ( )
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>

Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>

Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>||||||)?(\s*{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 97, n_remaining = 8095, next token: 198 '
'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 119
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 120, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7963, n_cache_tokens = 7963, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 27 (<)
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<
Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<
Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>||||||)?(\s*{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 98, n_remaining = 8094, next token: 27 '<'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 120
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 121, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7964, n_cache_tokens = 7964, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"<tool_call>\n"}}],"created":1754093086,"id":"chatcmpl-WwOVdpVGEkfJB5DuAkNO9Q8P3hbDPBpO","model":"qwen3-coder","system_fing

Grammar still awaiting trigger after token 1688 (function)
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function
Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function
Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>||||||)?(\s*{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 99, n_remaining = 8093, next token: 1688 'function'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 121
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 122, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7965, n_cache_tokens = 7965, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 22086 (=b)
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=b
Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=b
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 100, n_remaining = 8092, next token: 22086 '=b'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 122
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 123, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7966, n_cache_tokens = 7966, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"<function=b"}}],"created":1754093087,"id":"chatcmpl-WwOVdpVGEkfJB5DuAkNO9Q8P3hbDPBpO","model":"qwen3-coder","system_finger

Grammar still awaiting trigger after token 988 (ash)
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash
Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash
srv send: sending result for task id = 2
srv send: task id = 2 pushed to result queue
slot process_toke: id 0 | task 2 | n_decoded = 101, n_remaining = 8091, next token: 988 'ash'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 123
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 124, front = 0
slot update_slots: id 0 | task 2 | slot decode token, n_ctx = 131072, n_past = 7967, n_cache_tokens = 7967, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"ash"}}],"created":1754093087,"id":"chatcmpl-WwOVdpVGEkfJB5DuAkNO9Q8P3hbDPBpO","model":"qwen3-coder","system_fingerprint":"

Grammar triggered on regex: '<function=bash>
'
srv update_chat_: Parsing chat message: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash>

Parsing input with format Hermes 2 Pro: I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash>

Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <<<

/root/llama.cpp/build/bin/libggml-base.so(+0x14ee8)[0x77bd3c1e8ee8]
/root/llama.cpp/build/bin/libggml-base.so(ggml_print_backtrace+0x1e4)[0x77bd3c1e92b4]
/root/llama.cpp/build/bin/libggml-base.so(+0x24e29)[0x77bd3c1f8e29]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa8e1a)[0x77bd72aa8e1a]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa8e85)[0x77bd72aa8e85]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa90d8)[0x77bd72aa90d8]
/root/llama.cpp/build/bin/llama-server(+0x3b50e)[0x57172b97550e]
/root/llama.cpp/build/bin/llama-server(+0x16ec1e)[0x57172baa8c1e]
/root/llama.cpp/build/bin/llama-server(+0xd757e)[0x57172ba1157e]
/root/llama.cpp/build/bin/llama-server(+0xd7c68)[0x57172ba11c68]
/root/llama.cpp/build/bin/llama-server(+0xd8ac4)[0x57172ba12ac4]
/root/llama.cpp/build/bin/llama-server(+0xec0f5)[0x57172ba260f5]
/root/llama.cpp/build/bin/llama-server(+0xb185a)[0x57172b9eb85a]
/root/llama.cpp/build/bin/llama-server(+0x6c16b)[0x57172b9a616b]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x77bd72c4624a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x77bd72c46305]
/root/llama.cpp/build/bin/llama-server(+0x7b221)[0x57172b9b5221]
terminate called after throwing an instance of 'std::runtime_error'
what(): Invalid diff: 'I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>
<function=bash' not found at start of 'I'll help you create a React.js Todo app with TypeScript using Hono backend (with SQLite) via Bun package manager.

First let's understand what needs building:

1 Frontend: Create todo application in react + typescript
2 Backend API server : Using Honno web framework
3 Database Layer - Local sqlite database with SQL.js or similar

Let me check the current project structure to better plan our approach:
<tool_call>

'
Aborted

</code>
</details> 

[polymorphicshade]

I have the exact same issue @shekohex is seeing.

My llama server docker command:

-hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL --jinja --chat-template-file /templates/qwen3-coder.template -ngl 99 --threads -1 --ctx-size 32684 --temp 0.7 --min-p 0.0 --top-p 0.80 --top-k 20 --repeat-penalty 1.05 --verbose --verbose-prompt --port 8000 --host 0.0.0.0

I'm using the chat template suggested by @pwilkin.

EDIT: I should also point out the difference here:

Working:

Write a file with python code that downloads a file from a server.

Not working (causes the issue):

Write a python script that scrapes https://www.yahoo.com, save it to disk called "run_me.py"

[pwilkin]

Okay, recreated it now. Will try to look at it tomorrow.