Skip to content

Tool-call parser misses Qwen3-Coder's bare <function=...> format (no <tool_call> wrapper) #95

Description

@pekkah

Symptom

Sending an Anthropic `/v1/messages` request with `tools` to `Qwen3-Coder-30B-A3B-Instruct-Q4_K_M` returns either an empty stream (streaming) or a single non-functional text block (non-streaming) — the tool is never surfaced as a `tool_use` content block.

Verified live against the host today:

  • Same request to Qwen3.6-35B-A3B-UD returns a proper `tool_use` block and goes through the full `tool_result` roundtrip cleanly.
  • Same request to Qwen3-Coder-30B-A3B-Instruct returns `content: [{type:"text", text:"<function=get_weather>\n"}]` with `stop_reason: "end_turn"` and no params populated.

Cause

`JinjaChatTemplate.ParseToolCalls` only recognises tool calls when they're wrapped in `<tool_call>...</tool_call>`. Inside the wrapper it handles BOTH JSON and `<function=>` XML payloads, but the wrapper itself is required:

```csharp
const string open = "<tool_call>";
const string close = "</tool_call>";
...
int start = rawOutput.IndexOf(open, pos, ...);
if (start < 0) {
plain.Append(rawOutput, pos, rawOutput.Length - pos);
break; // no wrapper → entire output is plain text
}
```

Qwen3-Coder is trained on a different conversation format and emits `<function=name><parameter=k>v` at the top level, NOT inside `<tool_call>` wrappers. `/v1/messages` therefore returns the raw `<function=...>` text instead of a structured `tool_use` block.

Second issue (only manifests in streaming): `AnthropicEndpoints.HandleStreaming.ProcessTextChunk` buffers up to `ToolCallOpenTag.Length - 1` bytes (= 10 bytes) waiting for a potential `<tool_call>` open tag. Short bare-function outputs (`<function=...>` is 11+ bytes but the model can stop after the closing `` early) can be swallowed when the trailing buffer is dropped on stream end.

Scope

  1. In `JinjaChatTemplate.ParseToolCalls`, when no `<tool_call>` wrapper is present, also scan for top-level `<function=...>` blocks. Extract them via the existing `ParseXmlToolCallBlock` helper.
  2. Update `AnthropicEndpoints.ProcessTextChunk` to recognise BOTH `<tool_call>` and bare `<function=` as a "buffer until close-tag" trigger, with the correct matching close.
  3. Add server tests:
    • Non-streaming with a Qwen3-Coder-shape output produces a `tool_use` content block.
    • Streaming with the same emits `content_block_start{type:tool_use}` + `input_json_delta` events.

Acceptance

  • `POST /v1/messages` against Qwen3-Coder with a registered tool returns a structured `tool_use` block and `stop_reason: "tool_use"`.
  • Streaming variant emits the equivalent `tool_use` events.
  • Existing Qwen3-style `<tool_call>{...}</tool_call>` paths unchanged (existing 7 tool-related server tests stay green).

Related

  • `src/SharpInference.Core/JinjaChatTemplate.cs:1255-1297` — `ParseToolCalls` (wrapper-only matching)
  • `src/SharpInference.Server/Endpoints/AnthropicEndpoints.cs` — streaming state machine
  • Qwen3-Coder model card: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct — describes the `<function=>` format

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions