Feature Description
Qwen 3.5 35B A3B and Qwen 3.6 27B do not emit tool_calls when used with mlx_lm.server because the tool parser auto-detection (_infer_tool_parser) does not match their chat template patterns.
Current state
tool_parsers/qwen3_coder.py exists but only activates for Qwen3-Coder models (chat template contains <tool_call>\n<function=)
- Regular Qwen 3.5/3.6 chat templates do NOT contain these markers
_infer_tool_parser() returns None → no tool parsing → models respond with empty content when tools are provided
Steps to reproduce
curl -s http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"/Users/micky/MLX/qwen-35b-a3b-8bit","messages":[{"role":"user","content":"What is the weather in Milan?"}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}],"max_tokens":80}'
Result: content: "" and no tool_calls in the response.
Expected behavior
Qwen 3.5/3.6 models should emit structured tool_calls via the existing tool parser infrastructure, or via a dedicated qwen3 parser.
Environment
- mlx-lm 0.31.3 (git main, 2026-05-21)
- macOS 26.5, Apple Silicon (M5 Max)
- Models: mlx-community/qwen-35b-a3b-8bit, qwen-27b-bf16
Related
Workaround / request
Either:
- Add Qwen 3.5/3.6 regular chat template patterns to
_infer_tool_parser()
- Allow users to specify
--tool-parser qwen3_coder (or similar) explicitly via CLI flag
- Document which models currently support tool calling and how to enable it
Feature Description
Qwen 3.5 35B A3B and Qwen 3.6 27B do not emit
tool_callswhen used withmlx_lm.serverbecause the tool parser auto-detection (_infer_tool_parser) does not match their chat template patterns.Current state
tool_parsers/qwen3_coder.pyexists but only activates for Qwen3-Coder models (chat template contains<tool_call>\n<function=)_infer_tool_parser()returnsNone→ no tool parsing → models respond with empty content when tools are providedSteps to reproduce
Result:
content: ""and notool_callsin the response.Expected behavior
Qwen 3.5/3.6 models should emit structured
tool_callsvia the existing tool parser infrastructure, or via a dedicatedqwen3parser.Environment
Related
Workaround / request
Either:
_infer_tool_parser()--tool-parser qwen3_coder(or similar) explicitly via CLI flag