Skip to content

Fix Qwen(3.5) thinking format for Qwen3.5 models #2020

Description

@Airead

What happened?

Summary

Fix the Qwen thinking format handling in the OpenAI completions provider to correctly toggle thinking mode for Qwen3.5 models.

Problem

When using mlx-community/Qwen3.5-9B-4bit with the pi coding-agent, toggling the thinking/reasoning mode does not work correctly. The previous implementation shared the same code path for Z.ai and Qwen, only sending a top-level enable_thinking parameter:

if ((compat.thinkingFormat === "zai" || compat.thinkingFormat === "qwen") && model.reasoning) {
    (params as any).enable_thinking = !!options?.reasoningEffort;
}

Qwen3.5 models require enable_thinking to also be set inside chat_template_kwargs for the thinking toggle to take effect.

Fix

Separate Z.ai and Qwen into distinct code paths. Qwen now sends enable_thinking both at the top level and inside chat_template_kwargs:

if (compat.thinkingFormat === "zai" && model.reasoning) {
    // Z.ai uses top-level enable_thinking: boolean
    (params as any).enable_thinking = !!options?.reasoningEffort;
} else if (compat.thinkingFormat === "qwen" && model.reasoning) {
    // Qwen uses both top-level and chat_template_kwargs for compatibility
    const enableThinking = !!options?.reasoningEffort;
    (params as any).enable_thinking = enableThinking;
    (params as any).chat_template_kwargs = { enable_thinking: enableThinking };
}

Steps to reproduce

  1. Configure a custom model using mlx-community/Qwen3.5-9B-4bit with thinkingFormat: "qwen"
  2. Start the pi coding-agent with this model
  3. Enable thinking/reasoning mode
  4. Send a message — thinking works as expected
  5. Toggle thinking mode off
  6. Send another message — the model still outputs thinking content, the toggle has no effect
"llama.local": {
    "baseUrl": "http://127.0.0.1:8000/v1",
    "apiKey": "sk-xxxxxxxxx",
    "api": "openai-completions",
    "models": [
    {
        "id": "Qwen3.5-9B-4bit",
        "name": "Qwen3.5-9B-4bit",
        "reasoning": true,
        "input": ["text", "image"],
        "compat": {
        "supportsDeveloperRole": false,
        "thinkingFormat": "qwen"
        },
        "contextWindow": 65536,
        "maxTokens": 32768,
        "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
    }
    ]
}

Expected behavior

No response

Version

0.57.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions