[BUG] LM Studio Qwen-next Tool calling totally broken in versions 3.37.0+

[Nordln]

Problem (one or two sentences)

After updating for the first time in a while, tool calling seems to have utterly broken for my setup. I use roo on code-server with local AI qwen3-next-80b from LLM Studio. This model usually works extremely well for me. After any roo update beyond 3.36.16, all tool calling fails.

Context (who is affected and when)

My personal code-server instance with me as the sole user.

Reproduction steps

Running a code-server container via Containerd, inside a Lima VM on a Mac Studio M2 Ultra 192GB, accessed over https via a private cloudflare tunnel. LM added via OpenAI-compatible model. LM Studio running on Baremetal on the Mac, accessed via the pass-through internal IP.

1. Update Roo inside code-server in the normal way via the extension tab.
2. Perform tool testing via test mode.

Code-server details
code-server: v4.101.2
Code: 1.101.2

LM Studio details
Runtimes:

• mlx-llm-mac-arm64-apple-metal-advsimd@0.36.1 (MLX)
• llama.cpp-mac-arm64-apple-metal-advsimd@1.64.0 (GGUF)

Expected result

Tool calling works as-expected

Actual result

All tool calling fails

Variations tried (optional)

Tried Roo versions above 3.37.0 and tool calling fails. Versions 3.36.16 and below work as expected.

App Version

3.37.0 and upwards

API Provider (optional)

LM Studio

Model Used (optional)

qwen3-next-80b (8bit, MLX)

Roo Code Task Links (optional)

My tool testing roo mode: tool-tester-export.yaml

Relevant logs or errors (optional)

I've analyzed this issue and identified the root cause.

Root Cause:
This issue was introduced in version 3.37.0 by commit d00d9edec ("feat: deprecate XML tool protocol selection, force native for new tasks"). This change:

1. Removed the UI option to select XML tool protocol
2. Changed resolveToolProtocol() in src/utils/resolveToolProtocol.ts to always return "native" for new tasks
3. Forces all models to use OpenAI-style native tool calling with strict: true and additionalProperties: false

Why it breaks Qwen models in LM Studio:
Looking at src/api/providers/lm-studio.ts, the handler uses native tools when toolProtocol !== "xml" (line 51). Since XML protocol is now never selected, native tools are always used. The native tool calling requires:

• strict: true on tool definitions
• additionalProperties: false on all object schemas
• Specific JSON Schema format that some models like Qwen may not properly support

Proposed Implementation Plan:

I see two possible approaches:

Option A - Remove strict mode for LM Studio (Recommended):
Modify src/api/providers/lm-studio.ts to NOT set strict: true for tools, since LM Studio models (especially Qwen) may not fully support OpenAI's strict mode requirements. This would involve overriding convertToolsForOpenAI in the LM Studio handler to set strict: false.

Option B - Add LM Studio-specific setting:
Add a new setting in LM Studio configuration (e.g., lmStudioDisableStrictToolMode) that allows users to opt-out of strict tool mode for their specific models.

I recommend Option A as it's simpler and LM Studio already handles tool calling differently from OpenAI proper - the strict mode requirements are primarily for OpenAI's API compliance.

Please mention @roomote in a comment saying which option you prefer, and I'll proceed with the implementation.

[Nordln]

Additional

I did not use the "LM Studio" model type in the model profile, since the model ID (usually "qwen/qwen3-next-80b") is not accepted. I have two versions loaded, "qwen/qwen3-next-80b@8bit" and "qwen/qwen3-next-80b@5bit". Neither of these model IDs are accepted. A curl request to the LM Studio API shows the model name I entered is correct:

~ % curl http://localhost:1234/v1/models
{
  "data": [
    {
      "id": "qwen/qwen3-next-80b",
      "object": "model",
      "owned_by": "organization_owner"
    }
  ],
  "object": "list"
}
% 

[daniel-lxs]

Hey @Nordln Thank you for reporting this, we are now using native tool calls, in LM Studio this is referred as function calling, I am not very familiar with it but can you check if function calling is enabled for this model in LM Studio?

[Nordln]

Hey @Nordln Thank you for reporting this, we are now using native tool calls, in LM Studio this is referred as function calling, I am not very familiar with it but can you check if function calling is enabled for this model in LM Studio?

Thanks for that. I somehow neglected to mention my LM Studio version above. I am on LM Studio v0.3.37. According to the LM Studio release notes, v0.3.6 introduces Function Calling / Tool Use.

LM Studio docs seems to explain tool calling and function calling as broadly the same thing (doc page), but do split native calling from default calling. Their doc's state "Models with Native tool use support will have a hammer badge in the app, and generally perform better in tool use scenarios."
The qwen3 next model appears to support native support tool (and function?) calling as indicated via the hammer badge (see image).

That said, a recent vLLM bug report does suggest the Qwen3 Next models may not be returning correct responses to function calls however: "Returning to the contents of the parameter list may result in non-standard characters, causing parameter retrieval to fail"