docs(llm-providers): explain the structured tool_calls requirement for local endpoints (#520)

[devin-ai-integration]

Summary

Closes #520.

Documents why local/self-hosted endpoints "return tool calls as text" and how to fix it on the inference server. Docs only — no code changes. The runtime half of #520 already landed in main (f6f9469, 6eec34d): a text-only turn is nudged back into a tool call instead of parking the agent. What was missing is guidance for the underlying cause, which is server configuration.

Strix is entirely tool-driven, so a turn only does work if the endpoint returns a native tool_calls field. When a server isn't configured to parse the model's tool tokens, they land in content as text:

<tool_call>{"name": "exec_command", "arguments": {"cmd": "nmap ..."}}</tool_call>
exec_command(cmd="nmap ...", timeout=180)

The Agents SDK correctly reads that as an ordinary assistant message, so no action ever executes. This is server configuration, not the model or Strix — and it can't be fixed client-side without parsing model text into shell commands, which we must never do.

Verified by reproducing it on real local servers and confirming the fix flips it:

server	config	result
llama-server b10152	--jinja (now default)	structured 6/6
llama-server b10152	--no-jinja	refuses: 500 tools param requires --jinja flag
llama-cpp-python 0.3.34	no tool chat format	text-form calls 6/6 (the #520 leak)
Ollama 0.32	tool-capable template	structured
Ollama 0.32	template without tool wiring	refuses with the same 500
OpenRouter	native	structured, or HTTP 404 if no tool-capable endpoint

A correctly configured endpoint either returns a structured call or rejects the request — it never leaks. So the new docs section gives the per-server settings: llama.cpp --jinja + a matching chat template and reasoning aligned/off; Ollama a tool-capable template, thinking off (it pushes the call into content), and num_ctx at 16k–32k so the tool schemas aren't truncated out of the prompt; vLLM --enable-auto-tool-choice with matching --tool-call-parser and --reasoning-parser.

Scope change

This branch previously carried a preflight tool-calling probe, per-model-family sampling defaults, and invalid-tool-call recovery. All of it has been dropped and the branch rebuilt as docs-only:

• The probe is redundant now that main nudges a text-only turn back into a tool call (f6f9469, 6eec34d) — that covers the user-visible symptom without paying a probe request on every local scan.
• The sampling defaults were a hardcoded model table copied from another agent's source. It was already inaccurate (it keyed on glm-4 where upstream gates on glm-4.6/4.7, and used 0.6 for all kimi-k2 where thinking/2.5 variants want 1.0) and would drift with every new model release.

Not carried over: catching ModelBehaviorError in _run_cycle. It's a genuinely distinct path from main's fix — a call that was emitted structurally but is invalid (hallucinated or miscased name, unparseable arguments) — and today it fails the agent or kills a non-interactive scan. Happy to raise that separately if wanted.

Link to Devin session: https://app.devin.ai/sessions/f74044feadc04f02b17d2ccfdc4df7d7
Requested by: @0xallam

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

Documents the structured tool_calls requirement for local inference endpoints.

• Explains why text-form tool calls are not executed.
• Provides configuration guidance for llama.cpp, Ollama, and vLLM.
• Recommends context-window, reasoning, and sampling settings for reliable tool calling.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
docs/llm-providers/local.mdx	Adds documentation for diagnosing and configuring structured tool calling on supported local inference servers.

_{Reviews (2): Last reviewed commit: "docs(llm-providers): explain the structu..." | Re-trigger Greptile}

[devin-ai-integration]

@greptile