Bug Description
When a request fails because input_tokens + max_tokens > context_length (a pure output-cap error, not an oversized conversation), the recovery path in agent/conversation_loop.py (~3446-3461) shrinks max_tokens to available_out - 64 and retries. On local vLLM (qwen3.6-35b-a3b-awq), real input-token count reported by the provider grows ~65 tokens on every subsequent retry of the same turn, canceling the fixed 64-token margin every time.
Result: every retry lands at context_length + 1, never converges, burns through all max_compression_attempts (shared with real history-compression), then fails with Context length exceeded: max compression attempts (3) reached. — implying the conversation was too big to compress, when compression never ran at all.
Actual Behavior — real trace, 2026-07-09, local Qwen 200k
Attempt 1: input 134,465, requested max_tokens 65,536, sum 200,001 (over by 1)
Attempt 2: input 134,530, requested max_tokens 65,471, sum 200,001 (over by 1)
Attempt 3: input 134,595, requested max_tokens 65,406, sum 200,001 (over by 1)
Attempt 4: input 134,660, requested max_tokens 65,341, sum 200,001 (over by 1) -> fails
Affected Component
Agent Core (conversation loop, context compression, memory)
Root Cause Analysis
agent/conversation_loop.py:3446-3461 computes safe_out = max(1, available_out - 64) and retries, assuming available_out stays valid for the next attempt. In the observed failures the next request's real input had already grown ~65 tokens relative to the value used to compute it, canceling the margin exactly every time. Symptom reproduced precisely; underlying cause of the input-count drift itself not fully isolated.
Proposed Fix
Tested locally against the trace above: grow the margin exponentially per retry instead of a fixed 64, e.g. safety_margin = 256 * (2 ** compression_attempts). Converges within the existing 3-attempt budget where the fixed margin never did.
Environment
Hermes 0.18.2 [111544d], Python 3.11.15, Ubuntu, provider custom:vllm-qwen, compression.threshold=0.55.
Bug Description
When a request fails because
input_tokens + max_tokens > context_length(a pure output-cap error, not an oversized conversation), the recovery path inagent/conversation_loop.py(~3446-3461) shrinksmax_tokenstoavailable_out - 64and retries. On local vLLM (qwen3.6-35b-a3b-awq), real input-token count reported by the provider grows ~65 tokens on every subsequent retry of the same turn, canceling the fixed 64-token margin every time.Result: every retry lands at
context_length + 1, never converges, burns through allmax_compression_attempts(shared with real history-compression), then fails withContext length exceeded: max compression attempts (3) reached.— implying the conversation was too big to compress, when compression never ran at all.Actual Behavior — real trace, 2026-07-09, local Qwen 200k
Attempt 1: input 134,465, requested max_tokens 65,536, sum 200,001 (over by 1)
Attempt 2: input 134,530, requested max_tokens 65,471, sum 200,001 (over by 1)
Attempt 3: input 134,595, requested max_tokens 65,406, sum 200,001 (over by 1)
Attempt 4: input 134,660, requested max_tokens 65,341, sum 200,001 (over by 1) -> fails
Affected Component
Agent Core (conversation loop, context compression, memory)
Root Cause Analysis
agent/conversation_loop.py:3446-3461computessafe_out = max(1, available_out - 64)and retries, assumingavailable_outstays valid for the next attempt. In the observed failures the next request's real input had already grown ~65 tokens relative to the value used to compute it, canceling the margin exactly every time. Symptom reproduced precisely; underlying cause of the input-count drift itself not fully isolated.Proposed Fix
Tested locally against the trace above: grow the margin exponentially per retry instead of a fixed 64, e.g.
safety_margin = 256 * (2 ** compression_attempts). Converges within the existing 3-attempt budget where the fixed margin never did.Environment
Hermes 0.18.2 [111544d], Python 3.11.15, Ubuntu, provider custom:vllm-qwen, compression.threshold=0.55.