Support tool_choice for OpenAI-compatible adapters and detect text-format tool calls as a fallback

[flowingboy]

Local OpenAI-compatible models (e.g. Qwen3.8-27B-8bit served via MLX DSpark) sometimes emit tool calls as literal <tool_call> text instead of native structured tool_calls. The harness currently has no fallback: the text is not parsed, tool_choice is not supported, and the agent loop treats a text-only reply as a normal completed turn — so the calls silently never execute and the user just sees raw XML in the transcript.

Background, proposed changes, and acceptance criteria

Background (reproduced in a real session)

Session 「商业情报国内外现状扫描」 (session-46125110-12f6-4646-97ec-6d98d7b063c4, provider mlxdspark / Qwen3.8-27B-8bit, api: openai-completions):

• Step 1: the model correctly emitted one native structured call (ask_user_question) → executed fine.

• Later steps: the model wrote 6 subagent calls plus a todo_write call as literal text blocks:

  <tool_call>
  <function=subagent>
  <parameter=description>…</parameter>
  <parameter=prompt>…</parameter>
  </function>
  </tool_call>
  

• The harness executed zero of them (tool/call event count = 1 for the whole session), and the turn ended as completed/max-tokens with the raw text displayed to the user.

Current state verified in the codebase:

• No <tool_call> text parsing anywhere in the repo.
• GenerateOptions has no tool_choice — documented as an explicit MVP cut in packages/llm/llm-deepseek/README.md and packages/llm/llm/README.md ("tool_choice is not mapped — not part of the core vocabulary").
• No grammar / response_format constrained decoding (GenerateOptions is temperature/maxTokens/stop only).
• The agent loop ends a step as completed when the assistant message contains no tool-call blocks (packages/core/agent-loop/src/agent.ts), and llm-retry only retries transport-level failures.

Proposed changes (can be split)

1. tool_choice support (smallest viable, preferred) — add tool_choice to GenerateOptions and map it through the DeepSeek and pi-ai (OpenAI-compatible) adapters. The official DeepSeek API and most OpenAI-compatible local servers (vLLM, llama.cpp, etc.) accept "required" / "auto". When tools are available, the agent loop can send tool_choice: "required" (configurable), forcing structured tool calls and eliminating text-format calls at the source. This MVP cut now has a real-world trigger.

2. Text tool-call fallback detection (enhancement) — when a step finishes completed with text that contains <tool_call> / <function= markers while tools were available, inject a corrective user message ("tool calls must be issued through the native structured protocol, never as text") and re-run the step instead of ending the turn. Executing parsed text calls should be at most an opt-in degraded mode, with a tool-name allowlist, JSON argument validation, and prompt-injection protection.

Acceptance criteria

• Reproduce the "6 parallel subagent text calls" scenario on mlxdspark / Qwen3.8-27B-8bit; with change 1 enabled the model returns native tool_calls on every step (verifiable via tool_choice: "required").
• With change 2 enabled, a text-format call is detected, a corrective re-run is triggered, and the call eventually executes.
• Update the documentation that currently notes tool_choice is not mapped.

User- / model-visible change

• Users: local-model sessions no longer show "calls became plain text and silently did not run"; the failure becomes an explicit correction or forced structured call.
• Models: guided/forced to use the native tool protocol when tools are available.

Test evidence

• Attach excerpts from the above session's session.jsonl.zstd (assistant messages containing <tool_call> text blocks; tool/call count = 1) as reproduction evidence.

---

Note for maintainers: happy to provide the full session transcript excerpt on request. Upvotes welcome if this matches your roadmap.

[weijiafu14]

Ground-truth measurement supporting your premise, so this feature request doesn't have to argue about what the current behavior is:

We fault-injected the real pi-ai accumulator with a completion whose content is literal tool-call text (Anthropic-style <invoke> markup — same channel-miss family as your <tool_call> case): it lands as a plain text block, no toolCall detected, and the turn ends as a normal completed reply (transport-classify-results.json, case invokeTagInContent). So the "no fallback" you describe is confirmed at the accumulator layer too — nothing anywhere in the chain rescues text-format tool calls today, and the agent loop's "text reply = done" interpretation follows from that.

Related family datapoint: even when the model DOES use native tool_calls but wraps the arguments in a markdown fence, the args silently empty out — filed upstream with a repro harness as earendil-works/pi#8858. Your ask (detect text-format calls + support tool_choice) is the bigger sibling of that fix: fence-unwrap is unambiguous, prose-parsing needs a real design (which syntaxes, how strict, what on false positives) — worth pushing as one upstream detector covering <tool_call>, <invoke> (#2158) and friends rather than per-syntax patches.