[Bug]: Tool-loop guardrails miss loops where args vary but the result never changes (incl. repeated multimodal results)

[Vissirexa]

Bug Description

The tool-loop guardrails (agent/tool_guardrails.py) key their repetition tracking on the tool-call signature (tool name + canonical args) or on classified failure. Two loop shapes slip through both:

1. Varying-args / fixed-result loop. A tool call that "succeeds" with different arguments on every call but keeps returning the same blocked/empty/error-page body. For ex. execute_code wrapping a web fetch against a source that consistently 404s or soft-blocks a scraper. The exact-failure counter never fires (failed is false), and the _no_progress tracker is keyed by signature and scoped to an idempotent-tool allowlist, so it never sees the loop either.
2. Repeated multimodal results. A vision tool result of shape {"_multimodal": True, "content": [...]} embeds a base64 image payload, so str(result) is unique on every call even when the meaningful content (a placeholder caption like "Image loaded into your context") is identical. Repetition is never detected.

Observed running a local model (qwen3.6-35b) in unattended/gateway sessions: one session burned its iteration budget re-fetching the same blocked page with slightly different arguments; another re-loaded the same image via a vision tool 6 times with zero guard activity.

Steps to Reproduce

1. Enable tool-loop guardrails (defaults are fine; optionally hard_stop_enabled: true).
2. Have the agent call a tool that returns the same substantial error/blocked-page body regardless of arguments (e.g. fetch a URL that always serves the same 404/soft-block page), with the model varying the query args each attempt.
3. Watch the loop run past every configured threshold with no warning appended to the tool results.
4. Variant: have the agent load the same image via a vision/multimodal tool repeatedly - same outcome.

Expected Behavior

After N calls returning substantially identical content, the guardrails should treat it like the other repetition axes: append a soft warning to the tool result, and (when hard_stop_enabled) halt the loop at a higher threshold.

Actual Behavior

No guard fires at any point; the loop only ends when the iteration budget runs out.

Affected Component

Agent Core (conversation loop, context compression, memory)

Messaging Platform (if gateway-related)

No response

Debug Report

Not attaching "hermes debug share" links for this one: it's a code-level behavior
gap in agent/tool_guardrails.py, fully reproducible from unit tests and not
environment-specific, and my recent logs contain private gateway-session content.
Confirmed the gap still exists on current main by inspection — there is no
repetition tracking keyed on result content, only on call signature and
classified failure. Happy to share specific redacted log excerpts on request.

System info (from "hermes debug share --local"):

version:      0.18.0 (2026.7.1)
os:           Darwin 25.4.0 arm64 (macOS 15, Apple Silicon)
python:       3.11.15
openai_sdk:   2.24.0
provider:     lmstudio (local OpenAI-compatible endpoint)
model:        Qwen3.6-35B-A3B-MLX-8bit
toolsets:     hermes-cli
memory:       built-in
gateway:      telegram

Operating System

macOS 15

Python Version

3.11.15

Hermes Version

0.18.0

Additional Logs / Traceback (optional)

Root Cause Analysis (optional)

No response

Proposed Fix (optional)

No response

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR

[Franchesco1999]

Confirming this on a production VPS deployment with a frontier model — not a local/small-model artifact.

Environment

• Hermes Agent 0.17.0
• Provider: kimi-coding (Moonshot), model kimi-k2.5
• Gateway: Telegram, unattended cron sessions
• Linux (Ubuntu 22.04)

Case 3: session_search returning empty results with varying queries

Same failure shape as your case 1, different tool. The agent hallucinated a set of session IDs, then called session_search 16 times with different queries — every call returned {"success": true, "results": [], "count": 0}. No guardrail fired at any point.

Metric	Value
session_search calls with empty result	16
api_call_count	286
message_count	44
API calls per message	6.5 (system avg across 37 sessions: 0.5)
Guardrail warnings emitted	0

Verified against state.db — no Tool loop warning / same_tool_failure / no_progress entries exist for that session.

One detail worth adding to the root cause

It isn't only that the counters fail to increment on an empty-but-successful result — they are actively reset. In after_call():

self._exact_failure_counts.pop(signature, None)
self._same_tool_failure_counts.pop(tool_name, None)

Every empty-but-successful call wipes whatever was accumulated. So even a mixed loop — some calls erroring, some returning empty — can never reach the same_tool_failure threshold, because each empty success clears the counter. The guardrail isn't just insensitive to this shape; it's structurally unable to fire.

Why this shape matters in practice

The trigger here was model hallucination: the agent invented identifiers, then spent 286 API calls searching for them. Hallucination plus a guardrail blind to "success without progress" is a compounding pair — the fabrication is what generates the loop, and the guardrail gap is what makes it expensive instead of harmless. In unattended cron sessions this burns the iteration budget with no operator present to notice.

Happy to provide more detail from state.db if useful. Not planning a competing PR — flagging the production data in case it helps with prioritization (currently P3).