API server: reasoning_content and reasoning_effort never reach OpenAI-compatible SSE stream

[dominicelayda]

Summary

When connecting Open WebUI (or any OpenAI-compatible frontend) to Hermes's API server with a DeepSeek V4 backend, two independent gaps prevent reasoning/thinking content from reaching the SSE stream. The reasoning_effort configured in config.yaml is also silently dropped before it hits the DeepSeek API.

Gap 1 — reasoning_effort + thinking never sent to DeepSeek API

File: agent/transports/chat_completions.py

The transport layer injects reasoning_effort as a top-level kwarg only for Kimi (line 272), TokenHub (line 287), and LM Studio (line 305). It injects extra_body.thinking only for Kimi (line 342). No DeepSeek branch exists. DeepSeek V4 requires:

{
  "reasoning_effort": "max",
  "extra_body": {"thinking": {"type": "enabled"}}
}

Additionally, extra_body.reasoning (line 353) is gated behind params.get("supports_reasoning", False), which comes from AIAgent._supports_reasoning_extra_body() (run_agent.py:3476). That method returns False for direct DeepSeek connections because of the guard at line 3501:

if "openrouter" not in self._base_url_lower:
    return False

The reasoning_effort value IS correctly read from config by GatewayRunner._load_reasoning_config() and passed as reasoning_config dict through to the agent. It just never reaches the wire.

Gap 2 — reasoning_content discarded in API server streaming pipeline

The agent correctly captures delta.reasoning_content from DeepSeek's streaming response (chat_completion_helpers.py). It fires agent._fire_reasoning_delta(text) which calls self.reasoning_callback(text) (run_agent.py:3038-3044).

Three missing pieces in gateway/platforms/api_server.py:

1. _create_agent() (lines 893-911) never passes reasoning_callback. AIAgent.__init__ accepts it (run_agent.py:382) and _fire_reasoning_delta dispatches through it — but the API server never wires it. Compare with stream_delta_callback (line 903) which IS wired.

2. _handle_chat_completions (lines 1218-1220) only registers three callbacks:

   • stream_delta_callback=_on_delta
   • tool_start_callback=_on_tool_start
   • tool_complete_callback=_on_tool_complete

   No reasoning_callback is created or passed.

3. _write_sse_chat_completion (line 1398) has no mechanism to emit delta.reasoning_content chunks. The SSE writer only handles:

   • ("__tool_progress__", payload) → custom event: hermes.tool.progress
   • Plain strings → delta.content

   The standard OpenAI format for reasoning content is:

   {"choices": [{"index": 0, "delta": {"reasoning_content": "..."}, "finish_reason": null}]}

   Open WebUI has supported rendering delta.reasoning_content in a collapsible "Thinking" panel since early v0.5.x (PR chore(models): refresh OpenRouter + Nous fallback lists #23001 fixed a rendering bug March 2026, but field recognition itself is older).

Suggested patches

Patch A — DeepSeek reasoning_effort + thinking injection

agent/transports/chat_completions.py, after the TokenHub block (~line 299):

# DeepSeek V4: top-level reasoning_effort + extra_body.thinking
provider_name = str(params.get("provider_name") or "").strip().lower()
is_deepseek = provider_name == "deepseek"
if is_deepseek:
    if reasoning_config and isinstance(reasoning_config, dict):
        if reasoning_config.get("enabled") is not False:
            effort = str(reasoning_config.get("effort") or "high").strip().lower()
            if effort in ("low", "medium"):
                effort = "high"
            elif effort == "xhigh":
                effort = "max"
            api_kwargs["reasoning_effort"] = effort
            extra_body["thinking"] = {"type": "enabled"}

Patch B — Wire reasoning_callback in API server

File: gateway/platforms/api_server.py

In _handle_chat_completions (~line 1162, after _on_delta):

def _on_reasoning(delta):
    _stream_q.put(("__reasoning__", delta))

In the _run_agent call (~line 1218), add:

reasoning_callback=_on_reasoning,

In _write_sse_chat_completion (~line 1398), add before the else branch:

elif isinstance(item, tuple) and len(item) == 2 and item[0] == "__reasoning__":
    reasoning_chunk = {
        "id": completion_id, "object": "chat.completion.chunk",
        "created": created, "model": model,
        "choices": [{"index": 0, "delta": {"reasoning_content": item[1]}, "finish_reason": None}],
    }
    await response.write(f"data: {json.dumps(reasoning_chunk)}\n\n".encode())

Verification

After both patches:

1. Send a message through Open WebUI → Hermes → DeepSeek V4
2. Verify API request includes reasoning_effort and extra_body.thinking
3. Verify SSE stream includes delta.reasoning_content chunks
4. Verify Open WebUI renders a collapsible "Thinking" panel with reasoning content

Full technical audit with streaming-path diagram and verified line numbers is available if needed.

[alt-glitch]

Related PRs attempting to fix reasoning streaming in api_server:

• feat(api-server): stream chat reasoning chunks #13401 (open) — streams reasoning chunks via SSE
• fix(api_server): stream reasoning tokens as hermes.reasoning.delta SSE events #11482 (open, dup of feat(api-server): stream chat reasoning chunks #13401) — reasoning.delta SSE events
• feat(api-server): add config-driven tool_progress_events and reasoning_content #20109 (open) — config-driven reasoning_content + tool_progress_events
• fix(api_server): stream reasoning.delta events over /v1/runs SSE #15169 (open) — reasoning.delta over /v1/runs SSE

Gap 1 (reasoning_effort not sent to DeepSeek) appears novel — none of the above PRs address the transport-layer provider branching in chat_completions.py. Gap 2 (reasoning_callback not wired in api_server) is addressed by multiple competing PRs above.