Symptom
Sending an Anthropic `/v1/messages` request with `tools` to `Qwen3-Coder-30B-A3B-Instruct-Q4_K_M` returns either an empty stream (streaming) or a single non-functional text block (non-streaming) — the tool is never surfaced as a `tool_use` content block.
Verified live against the host today:
- Same request to Qwen3.6-35B-A3B-UD returns a proper `tool_use` block and goes through the full `tool_result` roundtrip cleanly.
- Same request to Qwen3-Coder-30B-A3B-Instruct returns `content: [{type:"text", text:"<function=get_weather>\n"}]` with `stop_reason: "end_turn"` and no params populated.
Cause
`JinjaChatTemplate.ParseToolCalls` only recognises tool calls when they're wrapped in `<tool_call>...</tool_call>`. Inside the wrapper it handles BOTH JSON and `<function=>` XML payloads, but the wrapper itself is required:
```csharp
const string open = "<tool_call>";
const string close = "</tool_call>";
...
int start = rawOutput.IndexOf(open, pos, ...);
if (start < 0) {
plain.Append(rawOutput, pos, rawOutput.Length - pos);
break; // no wrapper → entire output is plain text
}
```
Qwen3-Coder is trained on a different conversation format and emits `<function=name><parameter=k>v` at the top level, NOT inside `<tool_call>` wrappers. `/v1/messages` therefore returns the raw `<function=...>` text instead of a structured `tool_use` block.
Second issue (only manifests in streaming): `AnthropicEndpoints.HandleStreaming.ProcessTextChunk` buffers up to `ToolCallOpenTag.Length - 1` bytes (= 10 bytes) waiting for a potential `<tool_call>` open tag. Short bare-function outputs (`<function=...>` is 11+ bytes but the model can stop after the closing `` early) can be swallowed when the trailing buffer is dropped on stream end.
Scope
- In `JinjaChatTemplate.ParseToolCalls`, when no `<tool_call>` wrapper is present, also scan for top-level `<function=...>` blocks. Extract them via the existing `ParseXmlToolCallBlock` helper.
- Update `AnthropicEndpoints.ProcessTextChunk` to recognise BOTH `<tool_call>` and bare `<function=` as a "buffer until close-tag" trigger, with the correct matching close.
- Add server tests:
- Non-streaming with a Qwen3-Coder-shape output produces a `tool_use` content block.
- Streaming with the same emits `content_block_start{type:tool_use}` + `input_json_delta` events.
Acceptance
Related
- `src/SharpInference.Core/JinjaChatTemplate.cs:1255-1297` — `ParseToolCalls` (wrapper-only matching)
- `src/SharpInference.Server/Endpoints/AnthropicEndpoints.cs` — streaming state machine
- Qwen3-Coder model card: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct — describes the `<function=>` format
Symptom
Sending an Anthropic `/v1/messages` request with `tools` to `Qwen3-Coder-30B-A3B-Instruct-Q4_K_M` returns either an empty stream (streaming) or a single non-functional text block (non-streaming) — the tool is never surfaced as a `tool_use` content block.
Verified live against the host today:
Cause
`JinjaChatTemplate.ParseToolCalls` only recognises tool calls when they're wrapped in `<tool_call>...</tool_call>`. Inside the wrapper it handles BOTH JSON and `<function=>` XML payloads, but the wrapper itself is required:
```csharp
const string open = "<tool_call>";
const string close = "</tool_call>";
...
int start = rawOutput.IndexOf(open, pos, ...);
if (start < 0) {
plain.Append(rawOutput, pos, rawOutput.Length - pos);
break; // no wrapper → entire output is plain text
}
```
Qwen3-Coder is trained on a different conversation format and emits `<function=name><parameter=k>v` at the top level, NOT inside `<tool_call>` wrappers. `/v1/messages` therefore returns the raw `<function=...>` text instead of a structured `tool_use` block.
Second issue (only manifests in streaming): `AnthropicEndpoints.HandleStreaming.ProcessTextChunk` buffers up to `ToolCallOpenTag.Length - 1` bytes (= 10 bytes) waiting for a potential `<tool_call>` open tag. Short bare-function outputs (`<function=...>` is 11+ bytes but the model can stop after the closing `` early) can be swallowed when the trailing buffer is dropped on stream end.
Scope
Acceptance
Related