harness(openai): prompt-guided (text-mode) tool calling for non-native models#55
Conversation
…e models
Models without native tool calling (OpenAiModel built with_native_tool_calling(false)
— many self-hosted / local OpenAI-compatible runtimes that 400 on the `tools`
param) can now still call tools: the wire client embeds the tool specs in the
system prompt as a small `<tool_call>{"name":..,"arguments":..}</tool_call>`
protocol and parses those blocks back into ToolCalls, so the agent loop sees tool
calls identically to the native path — no harness change.
- new prompt_tools module: tool_instructions / with_tool_instructions (embed the
protocol on the system prompt) + parse_tool_calls_from_text / apply_to_response
(recover <tool_call> blocks into message.tool_calls, strip them from the prose).
- transport::translate_request: when profile.tool_calling is false and the request
carries tools, embed the protocol and send NO native `tools` on the wire.
- OpenAiModel::invoke + stream: recover tool calls from the completed response
(unary + terminal stream Completed item). Streamed text deltas still carry the
raw markup — cleaning them mid-stream is a follow-up.
The `<tool_call>` convention matches the long-standing OpenHuman host format so
models already prompted for it behave identically. 10 unit tests; full
providers::openai suite (75) green; fmt + clippy clean.
Unblocks the OpenHuman inference cutover of LOCAL runtimes to crate-native models
(issue #4249 Phase 3) — they relied on the host's prompt-guided tool parsing.
Claude-Session: https://claude.ai/code/session_01U8NDGbt9tKj443VquzycLB
📝 WalkthroughWalkthroughThe OpenAI provider adds prompt-guided tool calling for models without native tool support. It injects tool schemas into messages, omits wire-level tools, parses tagged tool calls from responses, and applies recovery to invoke and streaming completion paths. ChangesPrompt-guided tool calling
Estimated code review effort: 3 (Moderate) | ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65eabc4ffc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // handed tools gets the tool protocol embedded in its system prompt and no | ||
| // native `tools` on the wire (many local runtimes 400 on `tools`). The | ||
| // model's `<tool_call>` blocks are parsed back in [`Self::invoke`]/stream. | ||
| let prompt_guided_tools = !self.profile.tool_calling && !request.tools.is_empty(); |
There was a problem hiding this comment.
ToolChoice::None means the model must not call any tool, but this condition ignores request.tool_choice: a request that carries tools for later use but disables them for this turn still gets tool instructions injected and invoke/stream will parse <tool_call> blocks, so the agent loop can execute a tool the caller explicitly forbade. Keep prompt-guided tooling disabled (and skip parsing) when request.tool_choice is None, or render an explicit no-tool instruction.
Useful? React with 👍 / 👎.
| let tools: Vec<ToolWire> = if prompt_guided_tools { | ||
| Vec::new() |
There was a problem hiding this comment.
After a text-mode call is parsed, the agent loop appends the assistant tool_calls and Message::Tool result before the next model turn; this branch only drops the top-level tools, while the same history is still passed through translate_message, which serializes native assistant.tool_calls and role: "tool" instead of the <tool_result> protocol promised in the prompt. For the non-native/local endpoints this path targets, the second request after any tool call can be rejected or the model never sees results in the format it was instructed to use. Convert prior assistant/tool messages into text-mode tags when prompt_guided_tools is active.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/harness/providers/openai/transport.rs (1)
688-700: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExtract the prompt-guided predicate to avoid drift across the three call sites.
!self.profile.tool_calling && !request.tools.is_empty()is duplicated here (asprompt_guided_tools), ininvoke(Line 1108), and instream(Line 1184). Request translation omits wiretoolsbased on this, and both response paths recover the tags based on it — if one copy is later edited (e.g. adding a config flag), the request and recovery paths silently disagree and prompt-guided calls stop being recovered. A singlefn prompt_guided_tools(&self, request: &ModelRequest) -> boolkeeps all three in lockstep.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/harness/providers/openai/transport.rs` around lines 688 - 700, Extract the duplicated prompt-guided condition into a shared Self::prompt_guided_tools(&self, request: &ModelRequest) -> bool helper. Replace the local predicate in request translation and the corresponding checks in invoke and stream, preserving the current behavior while ensuring wire-tool omission and response tag recovery use the same decision.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/harness/providers/openai/transport.rs`:
- Around line 688-700: Extract the duplicated prompt-guided condition into a
shared Self::prompt_guided_tools(&self, request: &ModelRequest) -> bool helper.
Replace the local predicate in request translation and the corresponding checks
in invoke and stream, preserving the current behavior while ensuring wire-tool
omission and response tag recovery use the same decision.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 446e6e83-23c8-4b85-8972-62bcee0d38ab
📒 Files selected for processing (3)
src/harness/providers/openai/mod.rssrc/harness/providers/openai/prompt_tools.rssrc/harness/providers/openai/transport.rs
1 participant
What
Adds prompt-guided (text-mode) tool calling to
OpenAiModelfor modelswithout native tool support (
with_native_tool_calling(false)— many self-hosted/ local OpenAI-compatible runtimes that reject the OpenAI
toolsparam with a400).
Previously the crate was native-tools-only: a model with
profile.tool_calling == falsestill hadrequest.toolssent as nativetools(which those runtimesreject), and there was no path to embed tools in the prompt or parse tool calls
from text — the doc comments referenced a "harness embeds tool specs in the
prompt" behavior that wasn't actually implemented.
Now, when
profile.tool_callingis false and a request carries tools:translate_requestembeds a## Tool Use Protocolblock on the systemprompt (
<tool_call>{"name":..,"arguments":..}</tool_call>convention + eachtool's name/description/JSON-Schema params) and sends no native
tools.invokeandstreamrecover the model's<tool_call>blocks from thecompleted response into
message.tool_calls(unary + terminal streamCompleted), stripping them from the prose — so the agent loop sees tool callsidentically to the native path, with no harness change.
New
prompt_toolsmodule holds the pure builders/parser (tool_instructions,with_tool_instructions,parse_tool_calls_from_text,apply_to_response).Why
Unblocks OpenHuman's inference cutover of local runtimes to crate-native
models (issue #4249 Phase 3): they relied on the host's prompt-guided tool
parsing (
TextModeProvider+ pformat), which the crate lacked. The<tool_call>convention matches the long-standing OpenHuman host format, so models already
prompted for it behave identically after the cutover.
Testing
block parsing, missing-args default, system-message insertion).
providers::openaisuite (75) green — native tool path unchanged.cargo fmt --check,cargo clippy --lib -- -D warningsclean.Follow-up
Streamed text deltas still carry the raw
<tool_call>markup (tool calls arerecovered from the terminal
Completed); cleaning the live deltas mid-stream isa follow-up.
https://claude.ai/code/session_01U8NDGbt9tKj443VquzycLB
Summary by CodeRabbit
New Features
Bug Fixes