Skip to content

Security: outbound chat messages bypass redact_sensitive_text in v0.13.0 (gateway/platforms/*.py) #23810

Closed

Description

@babyspectrum

Summary

The redact_sensitive_text() function in agent/redact.py ships
correct regex patterns for API keys, tokens, and credentials (sk-*,
ghp_*, github_pat_*, Telegram bot<digits>:<token>, AWS, JWTs,
DB connection strings, etc.). It is correctly applied to log records
via RedactingFormatter and to tool output before it enters the LLM
context. However, it is NOT applied to LLM-generated responses being
delivered to messaging platforms (Telegram verified directly; Discord
and Slack likely affected by the same code path).

The gateway startup banner logs:

INFO gateway.run: Secret redaction: ENABLED (tool output, logs, and chat responses are scrubbed before delivery)

This is misleading: "chat responses are scrubbed before delivery" is
true only of log records of chat responses (via the RedactingFormatter
on log handlers), not of the actual delivered message bytes.

Sibling to #17691 (HERMES_REDACT_SECRETS default-OFF). That issue
addressed the default flag; this issue is about the function's CALL
SITE coverage on the gateway outbound path.

Reproduction

Hermes Agent v0.13.0 (v2026.5.7), Docker container.

config.yaml:

security:
  redact_secrets: true

unauthorized_dm_behavior: ignore

platform_toolsets:
  telegram: [messaging]
  # others: []

platforms:
  telegram:
    extra:
      disable_link_previews: true

/path/to/secrets.env:

TELEGRAM_BOT_TOKEN=<your-bot-token>
TELEGRAM_ALLOWED_USERS=<your-user-id>
TELEGRAM_HOME_CHANNEL=<your-user-id>

Start gateway: hermes gateway run. Gateway connects to Telegram and
logs Secret redaction: ENABLED (tool output, logs, and chat responses are scrubbed before delivery).

From your allowlisted Telegram account, DM the bot a block asking it to
echo back five synthetic token-shape lines, one per pattern under test:

  • LINE1: a sk- + 20 random hex chars (OpenAI/OpenRouter pattern)
  • LINE2: a github + _pat_ + 82 random [A-Za-z0-9_] chars (GitHub fine-grained PAT)
  • LINE3: a gh + p_ + 36 random [A-Za-z0-9] chars (GitHub classic PAT)
  • LINE4: a bot + 10 digits + : + 40 random [-A-Za-z0-9_] chars (Telegram bot token)
  • LINE5: a sk- + or-v1- + 73 random [A-Za-z0-9] chars (OpenRouter v1 key)

(Prefixes intentionally broken in this issue body so GitHub secret
scanning doesn't reject the post; reconstruct them exactly when
running the reproduction. Use synthetic placeholder bodies, never
real tokens.)

Expected

Bot reply contains each LINE wrapper but token bodies are masked
(e.g. sk-tes...cdef, or ***).

Actual

Bot reply contains all 5 token shapes verbatim, unredacted. Gateway
log confirms Sending response (414 chars) matching the unredacted
payload size.

The session JSONL transcript at /opt/data/sessions/<session>.jsonl
records the assistant content with the tokens still verbatim, i.e.
the LLM emitted them (which is fine; LLMs echo what users ask), but
the gateway delivered them as-is to Telegram with no scrubbing pass.

Code path

Inspected at upstream tag v2026.5.7:

  • gateway/platforms/base.py: no redact_sensitive_text call, no
    from agent.redact import.
  • gateway/platforms/telegram.py: no redact_sensitive_text call
    before bot.send_message / application.bot.send_message. Text
    transformation in this path is limited to _escape_mdv2,
    _strip_mdv2, and _wrap_markdown_tables (formatting, not
    redaction).

By contrast, RedactingFormatter in agent/redact.py correctly
wraps log handler output, so the same content IS scrubbed when
written to /opt/data/logs/*.log.

Suggested fix

In gateway/platforms/base.py (or wherever the canonical outbound
delivery happens), wrap the outbound text through
agent.redact.redact_sensitive_text(text) after Markdown formatting
and before the platform-specific send call:

from agent.redact import redact_sensitive_text

# ... existing formatting (markdown escape, table wrap, etc.)
text = redact_sensitive_text(text)
await self._platform_send(chat_id, text)

Or apply per-platform if scoping is needed (some operators may want
raw output on internal Slack DMs, etc.). Either way, the call site
should match what the startup banner already claims.

Alternatively, if the existing call path is intentional and the
banner is incorrect, the fix is to update the banner text to clarify
that only logs + tool output are scrubbed, and add a separate config
flag (e.g. security.redact_outbound_chat: true) to opt into
outbound scrubbing for operators who want it.

Impact

For deployments using Telegram/Discord/Slack as the primary delivery
channel for AI agent output (i.e. the documented v0.13 use case), any
token-shape string that reaches the LLM's response, whether echoed
back from a user prompt, accidentally surfaced from a skill that read
a credential file, or hallucinated as part of LLM output, is
delivered to the user verbatim. Gateway operators who rely on the
Secret redaction: ENABLED log line will infer a security posture
that doesn't match reality.

For our deployment we've shipped with a documented gap and
audience-of-one mitigation; we'd prefer a defence-in-depth fix
upstream.

Versions

  • nousresearch/hermes-agent:v2026.5.7 (= v0.13.0 "Tenacity Release")
    multi-arch image digest
    sha256:9462082533e603fc6ba4259c51d7722dcd8cbb0f348488ceedc35d06fa8f9324
  • Docker, Linux amd64
  • python-telegram-bot via upstream deps

Workaround

For now, deployments with this gap can:

  1. Restrict TELEGRAM_ALLOWED_USERS and TELEGRAM_HOME_CHANNEL to a
    single trusted operator (audience-of-one mitigation, leak still
    happens but only to the operator).
  2. Audit every enabled skill for tool-output paths that could surface
    a secret to LLM context.
  3. Patch locally by overriding BasePlatform.send_message (or
    equivalent) to wrap text through redact_sensitive_text.

Happy to send a PR if there's interest. Pointers on which file should
own the call site (base, per-platform, or a separate middleware
layer) would help shape the diff.

Activity

  1. sheepdestroyer commented on May 11, 2026

    @sheepdestroyer

    Triaged: Marked as a P1 security issue affecting the gateway component. Sensitive information can be inadvertently leaked to the end user by the gateway.

  2. added
    type/securitySecurity vulnerability or hardening
    P1High — major feature broken, no workaround
    comp/gatewayGateway runner, session dispatch, delivery
    comp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint
    on May 11, 2026
  3. babyspectrum commented on May 12, 2026

    @babyspectrum
    Author

    Thanks for the rapid P1 triage.

    Repro is still valid on v2026.5.7 (image digest sha256:9462082533e603fc6ba4259c51d7722dcd8cbb0f348488ceedc35d06fa8f9324) as filed; we have not regressed locally and are still relying on the audience-of-one mitigation (TELEGRAM_ALLOWED_USERS restricted to a single operator) plus a defence-in-depth redactor in our own MCP tool-output layer.

    Happy to send a PR. Could you nominate the preferred call site before I draft?

    1. Single hook in gateway/platforms/base.py (e.g. a _prepare_outbound(text) helper called from each platform's send path after Markdown formatting). Matches what the startup banner already claims, one source of truth, lower drift risk.
    2. Per-platform in each gateway/platforms/<platform>.py send method. More explicit, easier to scope per channel, but easier to miss a platform on future additions.

    My default would be (1) with a security.redact_outbound_chat config flag (default-on, matching the existing banner) so operators with legitimate raw-output needs on internal channels can opt out. Will hold the PR until you confirm the layer.

  4. leavedrop commented on May 24, 2026

    @leavedrop

    Hi @babyspectrum — I had filed #31423 which was correctly identified as a duplicate of this. Closing mine and bringing the pattern-detail discussion here, in case any of these patterns are missing from PRs #23821 / #24303.

    Patterns I'd suggest covering (with the prefix-anchored regex form to keep false-positive rate near zero):

    Provider / source Pattern
    Anthropic API key \bsk-ant-(api03-)?[a-zA-Z0-9_\-]{50,}\b
    OpenAI API key \bsk-(proj-)?[a-zA-Z0-9_\-]{40,}\b
    GitHub PAT (fine-grained + classic) \bgh[pous]_[a-zA-Z0-9]{36,}\b
    GitHub refresh token \bghr_[a-zA-Z0-9]{50,}\b
    Slack bot / user / app tokens \bxox[baprs]-[0-9]{10,}-[0-9]{10,}-[a-zA-Z0-9]{20,}\b
    AWS access key ID \bAKIA[0-9A-Z]{16}\b (+ associated aws_secret_access_key)
    Google API key \bAIza[0-9A-Za-z\-_]{35}\b
    Stripe live key \bsk_live_[0-9a-zA-Z]{24,}\b, \bpk_live_[0-9a-zA-Z]{24,}\b
    Generic env-line `\b[A-Z][A-Z0-9_]+_(API_KEY

    Replacement: <REDACTED-{label}> so a downstream agent reading the message recognizes the placeholder rather than parsing the leftover digits.

    A few practical notes from running similar redaction in production for a few weeks:

    1. False-positive rate on prefix-anchored patterns above: essentially zero. The only real catches were genuine secret-shaped strings, never test data or rotation docs.
    2. Generic env-line catches the long tail — even when a previously-unknown provider's key format isn't on the list, an LLM-quoted SOMETHING_API_KEY=... line gets caught.
    3. Opt-out kwarg is worth keeping for the rare legitimate case (security docs, key rotation runbooks, secret-scanning tutorials).

    If the patterns aren't already in #23821 / #24303, happy to PR them as an addition. If they are, this comment is just for the record.

    Thanks for landing this work.

  5. teknium1 commented on Jun 28, 2026

    @teknium1
    Collaborator

    Fixed on main in #53907 (1207d81ee).

    Your diagnosis was precise. The "total bypass" you reported against v0.13.0 was closed since — outbound LLM responses now route through _sanitize_gateway_final_response. But that path used a divergent narrow pattern set (_GATEWAY_SECRET_PATTERNS, 6 regexes) instead of the authoritative redact_sensitive_text, so it under-redacted vs the banner's promise. Two of your five test shapes still leaked: the GitHub fine-grained PAT (github_pat_…) and — most damning — the Telegram bot-token shape (bot<digits>:<token>).

    The fix makes _redact_gateway_user_facing_secrets delegate to redact_sensitive_text(force=True) — the same Tirith-grade redactor already applied to logs, tool output, and approval-command prompts — so the outbound chat path masks the full credential set the banner claims. force=True honors redaction even with security.redact_secrets off. The narrow local set is kept as a fail-soft second pass. Regression test parametrizes all 5 shapes you listed × every chat surface and confirms the secret body never reaches the user while prose survives.

    Thanks for the thorough, reproducible report with synthetic shapes — it mapped directly to the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High — major feature broken, no workaroundcomp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpointcomp/gatewayGateway runner, session dispatch, deliverytype/securitySecurity vulnerability or hardening

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions