qwen3-coder:latest does not support tools

[mihaim]

As per title, can't make qwen3-coder work under Ollama due to the tool error.

What to do about it?

[ReidBraswell]

https://x.com/thdxr/status/1952724502799843431

[GriffinMeyer]

I also had the same issue with Ollama, the LM studio beta fixed it!

[SiddharthPant]

Use lm-studio latest beta version to run the coder locally instead of Ollama. It has fixes for Qwen3-coder

[thdxr]

^ yes

[desugar-64]

It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now.

[johan-lejdung]

It works well with the latest beta of LM Studio, which I’m already using with OpenCode. For me, it works very well, I’m using it right now.
@desugar-64
What kind of performance are you getting? Mine is abysmal. I'm sitting on a good CPU, Radeon RX 6750 XT and 64GB ram. I'm wondering if I have to do anything special to tune performance

[desugar-64]

@johan-lejdung Hi.
I’m using a tuned model from Unsloth. It’s faster and more memory-efficient than the original, running with Q8XL UD quantization, a 127000-token context window, and generating around 17–20 tokens per second. With a smaller context window (e.g., 65536 tokens), performance ramps up to about 30 tokens per second.

My machine:

• CPU: AMD Ryzen R9 9950X3D
• RAM: 64GB DDR5 6400
• GPU: Nvidia RTX 5070, 12GB VRAM

The model’s experts are forced to run on the CPU, but the KV cache is on the GPU using Flash Attention and Q8 quantization.

[desugar-64]

To tune performance, it definitely helps to move the KV cache (basically your chat history) into GPU VRAM. If it doesn’t fit, you can either quantize it or reduce the context window size. As for the model’s experts, they’re forced to run on the CPU — but since they’re relatively small, the CPU can handle them quite well.

But, performance is a relative measure, it depends on what you consider abysmal. For instance, 20 tokens per second is perfectly fine for my development. It outputs text faster than I can read it, which is good enough to me. The most time-consuming part is prompt processing: when you add huge chunks of code into the chat history, it takes some time. But moving the KV cache to the GPU speeds this up.

[johan-lejdung]

@desugar-64

Thank you very much! Well for me using qwen3-coder with default settings and opencode it took around 30-60sec to even start generating tokens, once it did it was fast enough. I'll try the one you suggested and see if it's good enough to run as a local refactoring agent :)

[desugar-64]

around 30-60sec

The delay you’re observing is due to prompt processing. OpenCode uses a fairly lengthy system prompt. You can see this in the LM Studios logs, which show how it processes the prompt sequentially. One way to speed this up is by moving the KV cache to GPU memory and quantizing it.

Alternatively, you can try a different coding agents. From my experience, OpenCode doesn’t perform very well with this particular model, it often gets stuck editing files for several dozen seconds. Crush, on the other hand, is much faster at file editing with this model. I’ve also found RooCode to be a good alternative; their system prompts aren’t as long, and the system is smart enough to continue iterative development efficiently.

[johan-lejdung]

@desugar-64 sorry to keep bothering you, but are you using a different prompt template?

I'm getting this error after spinning it up on my desktop and then trying to access it on my laptop.

Did you use the template from here? https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#tool-calling-fixes

[desugar-64]

Ah, yes, that error. For some reason, the default prompt template is broken, LMStudio doesn’t understand it. I learned about this from GosuCoder's YouTube video. He provided a fixed version of the template, and it worked really well for me.
Here’s the fixed version I use:

{#- Copyright 2025-present the Unsloth team. All rights reserved. #}
{#- Licensed under the Apache License, Version 2.0 (the "License") #}
{#- Edits made by Unsloth to fix the chat template #}
{% macro render_item_list(item_list, tag_name='required') %}
    {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
        {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
            {{- '[' }}
                {%- for item in item_list -%}
                    {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
                    {%- if item is string -%}
                        {{ "`" ~ item ~ "`" }}
                    {%- else -%}
                        {{ item }}
                    {%- endif -%}
                {%- endfor -%}
            {{- ']' }}
        {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
    {%- endif %}
{% endmacro %}

{%- if messages[0]["role"] == "system" %}
    {%- set system_message = messages[0]["content"] %}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {%- set loop_messages = messages %}
{%- endif %}

{%- if not tools is defined %}
    {%- set tools = [] %}
{%- endif %}

{%- if system_message is defined %}
    {{- "<|im_start|>system\n" + system_message }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
    {%- endif %}
{%- endif %}
{%- if tools is iterable and tools | length > 0 %}
    {{- "\n\nYou have access to the following functions:\n\n" }}
    {{- "<tools>" }}
    {%- for tool in tools %}
        {%- if tool.function is defined %}
            {%- set tool = tool.function %}
        {%- endif %}
        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
        {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
        {{- '\n<parameters>' }}
        {%- for param_name, param_fields in tool.parameters.properties|items %}
            {{- '\n<parameter>' }}
            {{- '\n<name>' ~ param_name ~ '</name>' }}
            {%- if param_fields.type is defined %}
                {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
            {%- endif %}
            {%- if param_fields.description is defined %}
                {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
            {%- endif %}
            {{- render_item_list(param_fields.enum, 'enum') }}
            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
            {%- for json_key in param_fields %}
                {%- if json_key not in handled_keys %}
                    {%- set normed_json_key = json_key | replace("-", "_") | replace(" ", "_") | replace("$", "") %}
                    {%- if param_fields[json_key] is mapping %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- else %}
                        {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
                    {%- endif %}
                {%- endif %}
            {%- endfor %}
            {{- render_item_list(param_fields.required, 'required') }}
            {{- '\n</parameter>' }}
        {%- endfor %}
        {{- render_item_list(tool.parameters.required, 'required') }}
        {{- '\n</parameters>' }}
        {%- if tool.return is defined %}
            {%- if tool.return is mapping %}
                {{- '\n<return>' ~ (tool.return | tojson) ~ '</return>' }}
            {%- else %}
                {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
            {%- endif %}
        {%- endif %}
        {{- '\n</function>' }}
    {%- endfor %}
    {{- "\n</tools>" }}
    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- endif %}
{%- if system_message is defined %}
    {{- '<|im_end|>\n' }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- for message in loop_messages %}
    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
            {{- '\n' + message.content | trim + '\n' }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
            {%- if tool_call.arguments is defined %}
                {%- for args_name, args_value in tool_call.arguments|items %}
                    {{- '<parameter=' + args_name + '>\n' }}
                    {%- set args_value = args_value if args_value is string else args_value | string %}
                    {{- args_value }}
                    {{- '\n</parameter>\n' }}
                {%- endfor %}
            {%- endif %}
            {{- '</function>\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.previtem and loop.previtem.role != "tool" %}
            {{- '<|im_start|>user\n' }}
        {%- endif %}
        {{- '<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>\n' }}
        {%- if not loop.last and loop.nextitem.role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- elif loop.last %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- else %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}
{#- Copyright 2025-present the Unsloth team. All rights reserved. #}
{#- Licensed under the Apache License, Version 2.0 (the "License") #}

[johan-lejdung]

Thank you!

I got it running but it is easily 2-3min to first token. I think my GPU isn't used at all because it's not CUDA supported. And CPU could have more threads. I'll give this another go once I've upgraded my hardware down the line.

Much appreciation with the help!