Managed MCP calls can repeatedly exhaust turn budget and keep Discord typing until retry succeeds

[mostlydev]

Summary

A Hermes Discord runner with a managed MCP search tool can get stuck in a long active turn where Discord continues to show the agent as typing, but no visible reply is posted. The logs suggest the model/tool loop keeps running until cllama returns budget/deadline errors, and an interrupt from a follow-up Discord message does not fully clean up the outstanding managed tool/MCP work.

This is not project-specific. The concrete evidence below came from one deployed pod, but the failure mode appears to be a generic interaction between:

• Hermes Discord turn lifecycle / typing indicator handling
• cllama managed tool budgeting and deadlines
• stdio MCP sidecar cancellation behavior
• a realtime search MCP tool that can take long enough to hit those limits

Environment observed

• clawdapus: v0.13.2
• cllama: v0.5.0
• Hermes runner image: ghcr.io/mostlydev/hermes-base:v2026.3.17-claw.2
• MCP stdio sidecar image: ghcr.io/mostlydev/claw-mcp-stdio:v0.12.0
• Managed MCP tool: Perplexity search via perplexity-mcp
• Model route: openrouter/minimax/minimax-m2.7
• Surface: Discord mention-triggered Hermes agent turn

Repro shape

1. Run a Hermes Discord agent with a managed MCP search tool exposed through the stdio MCP wrapper.
2. Ask the agent to perform realtime search checks.
3. While the turn is still active, send another Discord mention asking for status.
4. Observe that the agent may remain typing for minutes and never post a visible response for that turn.

Evidence from one observed pod

Discord/user timeline, local time:

18:35:14 user asked the agent to run realtime Perplexity/search checks
18:37:37 user asked "Any luck?" while the agent was still shown as typing

Hermes/Discord runner log showed a follow-up message interrupting an active session:

[Discord] New message while session ... is active - triggering interrupt
⚡ Interrupted during API call.

cllama logs for the same window showed repeated model requests and 120s-ish failures:

2026-04-28T22:35:17Z request ... model=openrouter/minimax/minimax-m2.7
...
2026-04-28T22:37:17Z error latency_ms=120685 status_code=502 error="managed tool budget exhausted"
2026-04-28T22:39:18Z error latency_ms=120809 status_code=502 error="context deadline exceeded"
2026-04-28T22:39:37Z error latency_ms=120018 status_code=502 error="managed tool budget exhausted"
2026-04-28T22:41:38Z error latency_ms=120729 status_code=502 error="context deadline exceeded"
2026-04-28T22:42:17Z error latency_ms=38346 status_code=502 error="context canceled"

The MCP sidecar logged late/unknown responses around the same failure:

claw-mcp-stdio ignored response for unknown id 32
claw-mcp-stdio ignored response for unknown id 66

Additional observations:

• The agent process stayed healthy.
• Memory pressure was not involved; the container was well below its memory limit.
• The Hermes session file for the affected turn contained the user prompt, but no assistant/tool messages for the turn.
• Restarting only the affected agent cleared the stuck state.

Expected behavior

• Managed MCP calls should have a bounded failure path that returns a structured error to the runner/agent quickly enough for the agent to report failure.
• Discord typing should stop when cllama returns managed tool budget exhausted, context deadline exceeded, or when the turn is interrupted.
• A follow-up Discord interrupt should cancel outstanding cllama managed-tool work and propagate cancellation through the MCP sidecar.
• Late MCP responses after expected cancellation should either be handled cleanly or logged in a way that distinguishes normal cancellation from protocol mismatch.

Possible fixes

• Clamp managed MCP/tool-loop budgets per turn and surface a structured tool error instead of allowing repeated 120s failures.
• Propagate Discord turn interruption through cllama and the stdio MCP sidecar so outstanding calls are actually canceled.
• Ensure Hermes/Discord typing lifecycle is finalized on managed tool budget exhaustion, deadline, and cancellation paths.
• Add telemetry that identifies which managed tool consumed the budget, e.g. perplexity.search rather than only the model route.
• Add a regression test using a slow or looping MCP tool plus a follow-up Discord interrupt.

Related

This is related to, but distinct from, #190. That issue covers Hermes Discord tool-only/final-message behavior and visible progress cards. This issue is about managed MCP budget/deadline/cancellation behavior causing a long active turn and stuck typing.

[mostlydev]

Diagnosis update

The initial report said the Discord turn appeared stuck. It eventually did respond, so the more accurate failure mode is: managed MCP calls can repeatedly exhaust the per-turn mediation budget, Hermes retries the Discord turn, and Discord keeps showing typing until a later retry succeeds or the turn is interrupted/restarted.

This does not look like Perplexity itself being slow or failing. The persisted cllama history shows the MCP calls were generally successful and fast, but the model repeatedly re-issued similar search calls instead of stopping and answering.

Relevant persisted cllama history summary:

22:37:17 error 502, total_rounds=6, prompt_tokens=193031
  rounds 1-3: repeated S&P 500 / NVDA / Fed searches, all 200, ~3.6-8.3s each
  round 4: Apple / Seagate / tariffs searches, all 200, ~4.8-6.4s each
  round 5: repeated S&P 500 / NVDA / Fed searches, all 200, ~3.9-5.5s each
  round 6: repeated S&P 500 / NVDA searches, all 200, ~4.8s each

22:41:38 error 502, total_rounds=7, prompt_tokens=223461
  rounds 1-7: repeated AAPL / NVDA / S&P 500 searches, all 200, ~2.5-6.7s each

22:48:13 error 502, total_rounds=7, prompt_tokens=208102
  rounds 1-4: same S&P 500 search repeated once per round, all 200
  round 5: same S&P 500 search repeated 4 times in one round, all 200
  round 6: NVDA search, 200
  round 7: MRVL/MU/STX/AVGO/WDC/SYM searches, all 200, some duplicated

22:51:11 ok 200, total_rounds=3
  two tool rounds; searches all 200
22:51:29 ok 200, total_rounds=1
  final response posted to Discord

Hermes runner log around the same window:

[Discord] New message while session ... is active - triggering interrupt
Interrupted during API call.
API call failed (attempt 1/3): InternalServerError
Error code: 502 - {'error': {'message': 'managed tool mediation timed out'}}
Request context: 4 messages, ~7,133 tokens, 18 tools
send discord: "Perplexity is working ..."  # eventually posted

MCP sidecar still showed late responses after cancellation/timeout:

claw-mcp-stdio ignored response for unknown id 32
claw-mcp-stdio ignored response for unknown id 66
claw-mcp-stdio ignored response for unknown id 110

Interpretation:

• The default maxRounds=8 may be too low for some complex tool workflows, but this incident is not simply "8 is too low." Several failures hit the 120s total managed-turn timeout at 6-7 rounds before max rounds.
• The model/tool loop did too much repeated search work. A regular operator request turned into repeated same-query calls across retries.
• cllama currently reports this as a generic budget/deadline failure, but the operator-facing symptom is long Discord typing plus eventual retry rather than an immediate useful error.

Potential product fixes to consider:

• Add duplicate managed-tool-call detection within one mediated turn, especially identical tool name + arguments repeated across rounds.
• Expose more actionable telemetry in normal logs: total_rounds, managed tool names, duplicate call count, and whether failure was max rounds vs total timeout vs per-tool timeout.
• Consider separate defaults for search-like MCPs: maybe higher total timeout, but also stricter duplicate-query prevention or a lower duplicate threshold.
• When a managed turn times out, return a structured error that lets Hermes post a short failure message instead of silently retrying the same loop for minutes.
• Treat late stdio MCP responses after cancellation as expected cancellation if appropriate, or include enough IDs/state to diagnose protocol leaks.