Skip to content

feat: add tool parser support for Qwen 3.5/3.6 (non-Coder) models #1293

Description

@TinyWorkshopDesign

Feature Description

Qwen 3.5 35B A3B and Qwen 3.6 27B do not emit tool_calls when used with mlx_lm.server because the tool parser auto-detection (_infer_tool_parser) does not match their chat template patterns.

Current state

  • tool_parsers/qwen3_coder.py exists but only activates for Qwen3-Coder models (chat template contains <tool_call>\n<function=)
  • Regular Qwen 3.5/3.6 chat templates do NOT contain these markers
  • _infer_tool_parser() returns None → no tool parsing → models respond with empty content when tools are provided

Steps to reproduce

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"/Users/micky/MLX/qwen-35b-a3b-8bit","messages":[{"role":"user","content":"What is the weather in Milan?"}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}],"max_tokens":80}'

Result: content: "" and no tool_calls in the response.

Expected behavior

Qwen 3.5/3.6 models should emit structured tool_calls via the existing tool parser infrastructure, or via a dedicated qwen3 parser.

Environment

  • mlx-lm 0.31.3 (git main, 2026-05-21)
  • macOS 26.5, Apple Silicon (M5 Max)
  • Models: mlx-community/qwen-35b-a3b-8bit, qwen-27b-bf16

Related

Workaround / request

Either:

  1. Add Qwen 3.5/3.6 regular chat template patterns to _infer_tool_parser()
  2. Allow users to specify --tool-parser qwen3_coder (or similar) explicitly via CLI flag
  3. Document which models currently support tool calling and how to enable it

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions