[Bug]: Hermes empty assistant messages

[flaviovargasbrandao]

Bug Description

On current main (226e8de82), the request builder appends
{"role": "assistant", "content": "", "tool_calls": []} to the message array
sent to the model, and it does this in a runaway loop. When the model returns
another empty, act-less response after seeing a transcript that already ends in
its own empty turns, the builder appends one more empty assistant message and
retries the request. In captured live traffic, requests end with as many as 12
consecutive empty assistant messages, growing by exactly one per request every
couple of seconds.

For open-weight models served over an OpenAI-compatible endpoint (llama.cpp in
our case), the chat template renders each of these as a completed empty
assistant turn. A model that sees itself having said nothing 4 to 12 times in a
row degrades badly. It stops emitting tool calls and instead produces short
intention statements like "I'll read the summaries now." with no tool_call, and
the turn ends. This repeats. The visible downstream symptoms are
identical-argument tool-call retry loops (the same read_file re-issued 13
times) and guardrail halts.

v0.18.0 (662426ec3), in the same environment, never produces these messages
and never shows the failure.

One corroborating detail. With enable_thinking templates, llama.cpp actually
rejects these requests outright with HTTP 400
("Assistant response prefill is incompatible with enable_thinking."), because
it treats any trailing assistant message as prefill. We had worked around that
400 with --no-prefill-assistant, which silenced the error and, as it turns
out, converted a loud failure into silent context poisoning. The 400 first
showed up for us on 0.18.2; 0.18.0 never triggered it.

Steps to Reproduce

Steps to Reproduce

1. Configure a custom provider pointing at a llama.cpp llama-server
   (OpenAI-compatible, stream: true, tools enabled). Model is Qwen3.6-35B-A3B
   (IQ3_XXS GGUF), with chat_template_kwargs.enable_thinking: true and
   --no-prefill-assistant on the server. Without that flag you get the HTTP
   400 above instead of the silent failure.
2. Run a multi-step task that uses delegate_task fan-out with async
   completions. Ours was "research 3 topics with subagents in parallel, then
   consolidate the summaries and write an article."
3. Put a transparent logging proxy between the agent and llama-server and record
   each /v1/chat/completions body.
4. Watch the message array as the async delegation completions come back.

It shows up within about four minutes on every run. 3 of 3 runs on main
reproduced it; 0 of 4 runs on 0.18.0 did, with the same prompt, model, provider
and config. The config.yaml is byte-for-byte identical across both versions
(_config_version: 33, no migration fires). The only thing that changes is the
checked-out code.

Expected Behavior

Expected Behavior

The conversation sent to the model should never contain assistant messages with
empty content and an empty tool_calls array, and an empty model response
should not be re-appended as a context message and retried without bound.

Worth noting: the pre-API sanitizer already drops the empty tool_calls key
(#58755), but the empty assistant message itself still reaches the wire.

Actual Behavior

Actual Behavior

Captured wire traffic from one failing session, 57 requests in total. 29 of
those 57 (51%) end with two or more consecutive empty assistant messages.

Here is the terminal runaway, one request roughly every two seconds, one empty
assistant message appended each time:

seq	time	msgs	trailing empty assistant
48	09:41:07	47	3
49	09:41:11	48	4
50	09:41:12	49	5
51	09:41:14	50	6
52	09:41:16	51	7
53	09:41:17	52	8
54	09:41:19	53	9
55	09:41:21	54	10
56	09:41:23	55	11
57	09:41:24	56	12

The literal tail of request #57, with 12 of these in a row:

{"role": "assistant", "content": "", "tool_calls": []}
{"role": "assistant", "content": "", "tool_calls": []}
{"role": "assistant", "content": "", "tool_calls": []}

Behavior metric, counting assistant messages that announce an action but carry
no tool_call (main agent only, measured from the session store):

• 0.18.0: 2% (2 of 115 across 4 sessions)
• main: 18% to 56% across 3 sessions, plus guardrail_halts and that 13x
  identical-argument read_file loop on a delegation summary file.

These empty messages are never persisted to the session store (zero found in
state.db for every session). They live only in the built request, which is
why the issue stays invisible unless you capture live traffic.

Affected Component

Agent Core (conversation loop, context compression, memory)

Messaging Platform (if gateway-related)

N/A (CLI only)

Debug Report

### Debug Report


--- hermes dump ---
version:          0.18.0 [662426ec] (2026-07-03)   # config byte-identical to the main @ 226e8de82 checkout used to reproduce
os:               Linux 6.17.0-40-generic x86_64
python:           3.11.15
openai_sdk:       2.24.0
profile:          default
model:            qwen36-35b-a3b-iq3xxs
provider:         custom
terminal:         local

features:
  toolsets:           terminal, file, web, code_execution, skills, session_search, memory, delegation, todo, clarify, browser, vision
  mcp_servers:        0
  memory_provider:    openviking
  gateway:            stopped
  platforms:          none
  skills:             215

config_overrides:
  agent.tool_use_enforcement: True
  compression.threshold: 0.85
  display.show_reasoning: True
--- end dump ---

(IPs, paths, credentials and unrelated local tooling redacted. Log tail omitted
for size and privacy. The full sanitized report and the wire capture are
available on request; they contain private prompt data, so they are not
attached wholesale.)

Operating System

Ubuntu 24.04

Python Version

3.11.15

Hermes Version

0.18.2

Additional Logs / Traceback

### Additional Logs / Traceback

Searched open and merged issues and PRs first, per CONTRIBUTING:

- #42503 describes this same symptom from the outside ("the agent appears to
  start work and then stop", post-tool progress-only turns getting treated as
  final). This report is the wire-level cause behind it: by the time that
  placeholder turn happens, the context already ends in a pile of empty
  assistant turns.
- #63200 is phantom `content: ""` assistant messages (with tool_calls) breaking
  DeepSeek. Same message-hygiene family, different shape.
- #11213 is the prefill/thinking 400 conflict. That error is this same artifact
  getting rejected loudly by llama.cpp instead of silently accepted.
- Open PRs #31175, #31582 and #47280 sanitize adjacent shapes (empty content
  with tool_calls, partial-stream empty turns). None of them covers the
  accumulating `content:"" + tool_calls:[]` messages captured here.

Root Cause Analysis

Root Cause Analysis

The append site is most likely where an empty final_response gets pushed as
{"role":"assistant","content":""} ahead of a follow-up request. With tool
definitions active, that serializes as content:"" , tool_calls:[]. Two things
make it a bug rather than a quirk. First, the accumulation is unbounded: 12
deep, one more per request every couple of seconds. Second, these messages
survive into requests after the #58755 normalization chokepoint, which exists
specifically to keep semantically-empty tool_call shapes off the wire. The
"tool_calls": [] still riding on every captured request tells you the append
happens after that sanitizer, or on a path that skips it. That ordering is the
best lead for finding it.

What we ruled out, all verified equal between 662426ec3 and 226e8de82:

• config.yaml byte-identical (both _config_version: 33).
• Tool schemas: same 53 tools on the wire, toolsets.py identical.
• Tool-result reassembly after read_file, tested trivial, large, and with
  fragmented-SSE tool-call accumulation: identical, content intact
  (truncated: false).
• Async delegation batch-complete message composition: identical. The good
  version also processed 8 async completions in one run without trouble.
• delegate_task background default: False in both.
• Memory-provider recall injection: disabled and verified absent on the wire,
  and the failure still persisted at 56%.
• Same llama-server process, build and model on every run. Only the agent code
  changed, via git checkout.

One note toward the "intentional design, not a gap" rubric: a single trailing
assistant message can be a legitimate continuation or prefill mechanism on some
providers, and this report does not claim that is a bug. The bug is the
unbounded accumulation and its survival past the #58755 chokepoint.

Proposed Fix

1. Find the append site that runs after (or skips) the [Bug]: repair_message_sequence creates empty tool_calls array, triggering DeepSeek v4 HTTP 400 (follow-up to #56980) #58755 normalization,
   and extend that chokepoint, or the append site itself, to drop assistant
   messages that have neither content nor tool_calls.
2. Break the retry cycle. When the model returns an empty response and the last
   context message is already an empty assistant turn, don't append another one
   before re-requesting.
3. Treat any trailing assistant message as prefill-sensitive. The HTTP 400 with
   thinking templates is the loud version of this same bug, and masking flags
   like --no-prefill-assistant shouldn't be a prerequisite for open-weight
   providers.

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR

[flaviovargasbrandao]

sent the PR #66509

[kerimala]

I reproduced a second occurrence of this failure on a current Hermes checkout with Moonshot/Kimi through an OpenAI-compatible gateway:

HTTP 400 invalid_request_error: assistant message at position 76 must not be empty

#66509 covers the early generic sanitizer. Two later wire paths can still create/send a null assistant message:

• ChatCompletionsTransport.convert_messages() strips internal codex_message_items after that early sanitizer, which can leave an empty assistant row in the final request.
• handle_max_iterations() constructs and sends its summary request directly, bypassing transport build_kwargs().

I opened #67399 as a focused follow-up. It checks the final provider-visible shape in both paths, preserves tool/reasoning/multimodal assistant messages, and merges newly adjacent user turns to maintain role alternation. The report and PR contain only sanitized provider details.

[ai-strategy-yado-riki]

Adding a reproduction for a distinct trigger that produces the same "empty assistant message" symptom, in case it's a sibling root cause worth tracking. Observed on hermes-agent 0.15.2 (we haven't been able to confirm whether current main still has this exact path, so treat the version caveat seriously).

Environment

• Provider: OpenRouter → moonshotai/kimi-k3 (a reasoning model), streaming enabled, tools enabled (Discord gateway runtime).
• Not llama.cpp / not --no-prefill-assistant — so this is a different setup from the original report, but the poisoned-transcript shape is the same.

Trigger (single empty from stream-drop recovery, not a runaway loop)

A mid-stream disconnect on a reasoning model:

WARNING agent.stream_diag: Stream drop on attempt 2/3 — retrying ... error_type=RemoteProtocolError
  error=peer closed connection without sending complete message body (incomplete chunked read) http_status=200
WARNING agent.chat_completion_helpers: Streaming failed after partial delivery, not retrying: ...
WARNING agent.chat_completion_helpers: Partial stream delivered before error; returning length-truncated
  stub with 0 chars of recovered content so the loop can continue from where the stream died: ...

Because the model is a reasoning model, the reasoning deltas had streamed (so deltas_were_sent is true) but zero content tokens arrived before the socket dropped.

In interruptible_streaming_api_call (chat_completion_helpers.py), the partial-recovery branch computes:

_partial_text = (getattr(agent, "_current_streamed_assistant_text", "") or "").strip() or None  # -> None

and then still returns a stub with content=_partial_text (i.e. content=None) and finish_reason="length". That empty assistant message gets appended to the transcript and persisted to state.db. On the next request, a strict provider (Moonshot/Kimi) rejects it:

Non-retryable client error: 400 - {"error":{"message":
  "Invalid request: the message at position 42 with role 'assistant' must not be empty", ...}},
  provider_name: "Moonshot AI"

Because it's persisted, every subsequent user turn in that thread re-sends the empty message and 400s again — the thread is stuck until the history is repaired.

Minimal reproduction (deterministic)

We had the exact failing request body captured (a dumped /v1/chat/completions payload whose messages[42] is {"role":"assistant","content":"","tool_calls":false}). Replaying it verbatim against moonshotai/kimi-k3:

• as-is → HTTP 400 "must not be empty"
• after padding the empty assistant content to a non-empty sentinel → HTTP 200 OK

So the poison is exactly the empty-content assistant turn, and any non-empty placeholder clears it.

Fix that worked for us (hotfix on 0.15.2, two layers)

1. Don't create the empty stub. In interruptible_streaming_api_call, when recovery yields _partial_text is None and there are no partial tool-call names, raise result["error"] instead of returning a content=None stub. With 0 content delivered there's no partial answer to preserve and no duplicate-token risk, so re-raising lets the outer retry / fallback / (empty)-terminal machinery handle it cleanly.
2. Defensive pre-send pad. Right before building the request kwargs (next to reapply_reasoning_echo_for_provider), replace any assistant message with empty/whitespace content and no tool_calls with the existing "(empty)" sentinel — on the send-only msg.copy() list so persisted history isn't mutated. This also heals sessions already poisoned before the fix.

This looks related to #31449 (partial_stream_recovery) and #31583 (empty content with tool_calls), but our case is empty content with no tool_calls, arising specifically from a reasoning model's stream dropping before any content token. Happy to share more log/payload detail if useful for repro.