Skip to content

OpenAI /v1/chat/completions: tool-call wire-format support (parity with /v1/messages) #97

Description

@pekkah

Background

`ChatCompletionRequest` has no `tools` / `tool_choice` field today, so the OpenAI endpoint silently drops any tool definitions the client sends and the model never sees them. `/v1/messages` (Anthropic) has full tool support; `/v1/chat/completions` does not.

OpenAI wire shape:

```json
{
"model": "...",
"messages": [...],
"tools": [{"type":"function","function":{"name":"get_weather","description":"...","parameters":{...}}}],
"tool_choice": "auto"|"none"|{"type":"function","function":{"name":"..."}}
}
```

Response:
```json
{
"choices":[{
"message":{
"role":"assistant",
"content": null,
"tool_calls":[{"id":"call_abc","type":"function","function":{"name":"...","arguments":"{...}"}}]
},
"finish_reason":"tool_calls"
}]
}
```

Streaming variant emits incremental `tool_calls` deltas (each `delta.tool_calls[i]` is an array element addressed by index).

History side: a `role:"tool"` message with `{tool_call_id, content}` echoes a prior result.

Scope

  1. Add `Tools`, `ToolChoice` to `ChatCompletionRequest`. Plus matching `tool_calls` field on `OaiAssistantMessage` and a new `ToolCall` record in the response shape. Register types in `SharpInferenceJsonContext`.
  2. Render tool definitions into the Jinja context (`tools` key) on the prompt side, same path the Anthropic endpoint already uses via `ChatTemplateRenderer.Format(dict)`.
  3. Use the new tool-call adapter (issue Pluggable tool-call translation layer: Anthropic wire format ↔ model-specific syntax #96) to parse model output → emit OpenAI's `tool_calls` array shape and set `finish_reason: "tool_calls"` when calls were produced.
  4. Accept history-side `role:"tool"` messages in `BuildMessageList` and pass them through to the template.
  5. Streaming: emit `delta.tool_calls` increments per chunk, matching OpenAI's spec (incremental name + JSON-fragment arguments).
  6. Tests: parallel coverage to the Anthropic suite (`ChatCompletion_WithTools_NonStreaming`, streaming variant, history-echo).

Acceptance

  • OpenAI-protocol tool-calling client (e.g. `openai` Python SDK, LiteLLM proxy) can do a tool-use roundtrip against `/v1/chat/completions`.
  • Streaming variant emits incremental `tool_calls` deltas in the documented shape.
  • Shares the parser layer from Pluggable tool-call translation layer: Anthropic wire format ↔ model-specific syntax #96 — no duplicate Qwen3 / Qwen3-Coder / Llama branching here.
  • Existing OpenAI tests + Anthropic tool tests stay green.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions