Repository navigation
Description
Bug Description
Context compaction summaries are leaking into chat channels as visible assistant messages. In affected sessions, 29 of 30 assistant messages were compaction summaries — the model became trapped in a compaction loop, repeatedly echoing the summary back as its own response.
A part of this was fixed in #20169 in response to issues
#11475
#14521
but the fix seems to be incomplete because it doesn't cover the case where the ordering of messages by random alternating chance ends up landing in role=assistant slot.
The proposed fix is to always override to role=user for compaction summaries because of how some LLMs put more weight on user input than their own output, and from there the existing fix in PR 20169 will take care of the rest.
Steps to Reproduce
-
Run a long-lived session that accumulates enough context to trigger compaction (observed with 1400+ messages, ~75K+ tokens)
-
Use a compression model that is the same as the main model on a shared llama.cpp instance (so compression competes for GPU time)
-
Compression request times out at the default 120s → falls back to deterministic summary
-
Fallback summary is inserted with role="assistant" (see root cause below)
-
Model sees the summary as its own prior output and reproduces it verbatim as its next response
-
Response adds to context → triggers compaction again → loop
Expected Behavior
-
Memory compaction should not have resulted in the summary being output back into the chat session.
-
When both LLM summary and detereministic summary failed, compaction should probably give up as from that point on it will only end up pointlessly burning more GPU without ever compacting the context until some external intervention happens.
Actual Behavior
A large message starting with:
[CONTEXT COMPACTION — REFERENCE ONLY]
is emitted to the chat stream, as "assistant" role.
Affected Component
Agent Core (conversation loop, context compression, memory)
Messaging Platform (if gateway-related)
N/A (CLI only)
Debug Report
Report https://paste.rs/X2Ldb
agent.log https://paste.rs/IrqoN
gateway.log https://paste.rs/7hg1HOperating System
EL10
Python Version
3.12.13
Hermes Version
0.16.0
Additional Logs / Traceback (optional)
Root Cause Analysis (optional)
Two bugs in agent/context_compressor.py:
Bug 1 — assistant role summaries lack the end marker (primary cause of leak)
The role selection logic at line ~1997 picks a role to avoid consecutive same-role collisions with head and tail messages. When last_head_role="assistant" AND first_tail_role="user", both roles collide, so it falls through to role="assistant".
PR #20169 added the --- END OF CONTEXT SUMMARY --- end marker to prevent the model from treating the summary as fresh instructions — but guarded it with and summary_role == "user". Summaries inserted as role="assistant" never get the marker, so the model echoes them as its own prior output.
# Line ~2012 — only "user" role gets the end marker:
if not _merge_summary_into_tail and summary_role == "user":
summary = summary + "\n\n--- END OF CONTEXT SUMMARY ---"
Bug 2 — Deterministic fallback doesn't count as ineffective (causes infinite loop)
When the LLM compression times out, the deterministic fallback produces minimal compression (e.g. 249→224 messages). The anti-thrashing logic at ~line 739 stops retrying after 2 ineffective compressions — but the fallback path never increments _ineffective_compression_count, so it keeps retrying indefinitely.
Proposed Fix (optional)
-
Mark fallback as ineffective so the anti-thrashing logic won't retry compression on the next turn
Possible part-mitigation for this is to increase auxiliary.compression.timeout from default 120 to something much larger when large contexts and/or slower hardware are used. -
Always use role="user" with end marker for memory compaction summary output
Are you willing to submit a PR for this?
- I'd like to fix this myself and submit a PR
Activity
GottZ commentedon Jul 28, 2026
Summary
Four PRs address #42768 by preventing compaction summaries from appearing as assistant output; #42783, #42804, and #42897 cover variants of the user-role change, while #42830 most completely combines the role invariant, unconditional end marker, fallback anti-thrashing, and regression coverage. The relevant fixes were subsequently implemented on main through #45153 and #52160/#56197, with current role handling preserving provider alternation constraints.
Related pull requests
partial— (+20/-30) — partial fix, closed in favor of fix(agent): keep compaction summaries as user role (#42768) #42830: forces user-role summaries, always appends the end marker, and marks deterministic fallback as ineffective, but provides no regression tests; it remains relevant as the focused precursor to fix(agent): keep compaction summaries as user role (#42768) #42830.partial— (+82/-100) — partial fix, duplicate of Fix memory compaction leak when role=assistant #42783/fix(agent): keep compaction summaries as user role (#42768) #42830: removes assistant-role summaries and updates tests, but does not add the deterministic-fallback anti-thrashing fix; contributors closed it during consolidation.best fix— (+188/-106) — best verified fix, but closed as superseded by main: it directly covers both reported causes with user-role summaries, an unconditional end marker, fallback anti-thrashing, and targeted regression tests. Despite its earlier positive verification, the later contributor review documents that fix(compression): append END OF CONTEXT SUMMARY marker to all standalone summaries (salvages #33346) #45153 and bug(adapter): HTTP 400 after double context compression — first message is assistant, not user #52160/fix(compressor): batch salvage \u2014 orphan tool_calls, cooldown abort flags, system-head summary role (#51225 #52056 #52167) #56197 already fixed the issue on main and that merging this stale branch would reintroduce consecutive-user behavior the current implementation avoids.partial— (+68/-41) — partial fix, closed as a duplicate of fix(agent): keep compaction summaries as user role (#42768) #42830: keeps summaries on the user side by merging at role collisions and adds related tests, but omits fix(agent): keep compaction summaries as user role (#42768) #42830's deterministic-fallback anti-thrashing coverage and complete marker invariant.Duplicates
#42783, #42804, #42830, and #42897 substantially overlap on preventing assistant-role compaction summaries; #42830 is the most complete historical implementation, while #42804 and #42897 are narrower duplicates and #42783 is its untested focused precursor.
Suggested consolidation
Do not reopen or merge #42830: although Verify selected it as the best fix in this cluster, the blocking contributor review explains from the evolved main diff that #45153 and #52160/#56197 already cover the leak and loop while preserving the current alternation invariant. Retain #42830 as the reference implementation, keep #42783, #42804, and #42897 closed as duplicates/superseded, and verify #42768 against current main before closing the issue as already fixed.
Complex graph
flowchart LR classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937 classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151 classDef best stroke-width:3px,stroke:#b45309 classDef target stroke-width:3px,stroke:#4338ca I42768(["issue #42768 (open)"]) subgraph Dup42783 ["PRs duplicating each other"] P42783["PR #42783 (closed)"] P42804["PR #42804 (closed)"] P42830["PR #42830 (closed)"] P42897["PR #42897 (closed)"] end P42783 -.->|partial| I42768 P42804 -.->|partial| I42768 P42830 -->|best fix| I42768 P42897 -.->|partial| I42768 class I42768 open class I42768 target class P42783 closed class P42804 closed class P42830 closed class P42897 closed class P42830 best click I42768 "https://github.com/NousResearch/hermes-agent/issues/42768" click P42783 "https://github.com/NousResearch/hermes-agent/pull/42783" click P42804 "https://github.com/NousResearch/hermes-agent/pull/42804" click P42830 "https://github.com/NousResearch/hermes-agent/pull/42830" click P42897 "https://github.com/NousResearch/hermes-agent/pull/42897"Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).
Cross-PR triage: Reviewed 4 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 52 kB of PR diffs, 16 kB of issue/PR text, 5 kB of discussion (9 comments), 8 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.
pastyGRB commentedon Aug 8, 2026
Related bug: compaction summary spliced into an existing
role="user"message (not just injected as a newrole="assistant"message)Filing as a comment on this issue since it's the same underlying subsystem (
agent/context_compressor.py) and the same class of failure (compaction summary text bleeding into the wrong conversational role), but the trigger and the concrete symptom are different from the original report, so I want to make sure the reproduction case doesn't get lost if a fix lands here that only addresses therole="assistant"injection path.What we were trying to do
User uploaded an image into an active session and asked for analysis of it (a vision request). This was a normal, single-turn "here's an image, please look at it" interaction — nothing unusual about the session up to that point.
What happened
The vision/analysis call on that image did not complete successfully (appears to have failed or timed out — we did not get a clean assistant reply attached to that turn). This left the session's tail in an unusual state: the last real message in the transcript was a lone, unanswered
role="user"message carrying the uploaded image, with no assistant response after it.Some time later, context compaction ran on this session (transcript was otherwise short). Because the tail was just
system prompt → single unanswered user (image) turn, the compactor's merge-into-tail logic decided it needed to merge the LLM-generated compaction summary into that trailing message rather than insert it as a new one.The result: the assistant-voice, third-person compaction summary ("Historical Task Snapshot / Completed Actions / Remaining Work…") got spliced directly into the content of the user's own image message, but the message's
rolestayed"user". From that point on, replaying/reading the session, it looks exactly like the user typed a paragraph describing "what needs to be done next" — i.e., the agent's own planning output got attributed to the human. This is worse than simple visible pollution: it silently rewrites the historical record of who said what.Because the session transcript remained small after this, compaction re-triggered on subsequent turns and stacked additional summary text onto the same user message across multiple passes — we saw this happen at least twice more in the same session (three affected messages total).
Why this is distinct from the original issue on this thread
role="assistant", which then get echoed back by the model in a loop. The suggested fix in this thread is "always forcerole="user"for injected compaction summaries."role="user"message. If the fix for this issue is "always coerce injected-summary role to user," that would not catch our case — arguably it makes the failure mode structurally more likely, since it removes the one signal (role="assistant") that could otherwise distinguish "text the agent generated" from "text the user actually typed."Root cause (our analysis)
In
agent/context_compressor.py, the compaction routine has a merge path — referred to here as_merge_summary_into_tail(name from our read of the logic; please forgive if the current symbol name differs slightly) — that is taken when the compactor determines the transcript tail is too sparse/short to insert the summary as its own standalone message (e.g., only one trailing message present, or roles that would otherwise "collide" if a new message were inserted). This path:contentfield.role.When the tail message being merged into happens to be a genuine, unanswered
role="user"message (as in our case — an image upload with no assistant reply yet), the merged result is a message that is 100% agent-authored content permanently stored underrole="user". There is no marker anywhere in the message distinguishing "original user content" from "injected compaction summary" — so any downstream consumer (including us, and presumably other users) reading the transcript has no way to tell this apart from something the human actually typed.Steps to reproduce
role="user"message containing only the image + request, with nothing after it.system → lone unanswered user/image message.content.roleis still"user", butcontentnow contains an assistant-voice summary (typically structured like "Historical Task Snapshot," "Completed Actions," "Remaining Work," etc.) glued onto/after the original image request.role="user"entry.Concrete instance we observed
20260808_155141_936d6fdfrole="user", attached to the user's uploaded-image turn.Suggested fix direction
Whatever change lands for the
role="assistant"injection case in this issue should also cover this merge-into-tail path:"user". If the compactor decides a standalone summary message can't be inserted because the tail is too sparse, it should still emit the summary as its own message (with clearly agent/system-attributed role or a dedicated metadata flag), rather than merging it into whatever message happens to be sitting there.role="user"message as a merge/attachment target — this is exactly the scenario where merging is most likely to corrupt the historical record, since there's no assistant turn to "absorb" the summary into safely.role="system"or ametadata.injected_by="compactor"flag) so they are provably distinguishable from both genuine user and genuine assistant turns, regardless of which merge path is taken. This would resolve both the originalrole="assistant"echo-loop issue and ourrole="user"corruption case with one consistent mechanism.Happy to provide further transcript excerpts or logs if useful.
Setup / environment details
hermes_agent.egg-info/PKG-INFO)hermes-agentrepo at build time (no local modifications to delivered agent code); custom config/env/skills are supplied via mounted volumes only.openai==2.24.0,httpx==0.28.1,pyyaml==6.0.3,tenacity==9.1.4,jinja2==3.1.6github_copilot/claude-sonnet-5via a custom OpenAI-compatible router provider (custom:router),supports_vision: truegithub_copilot/gpt-5.4(same router)context_length: 200000incustom_providersmodel map); fallbackgpt-5.4is configured at 256,000compression:block inconfig.yaml):enabled: truethreshold: 0.5(compaction triggers at 50% of context budget)target_ratio: 0.2(compress down to 20% of budget)protect_last_n: 20,protect_first_n: 3max_turns: 60,reasoning_effort: medium,verify_on_stop: falseagent/context_compressor.pyor any other delivered file — this is 100% stock behavior on a clean pull of the repo.Image being analyzed is provided below for context.
fix(agent): guard merged assistant compaction handoffs
kanban: resume-on-retry session continuity for re-review rounds (#1)
fix(agent): guard merged assistant compaction handoffs
fix(agent): guard merged assistant compaction handoffs