Skip to content

Delivered-text persistence surfaces bypass PII/redaction hooks (rich-sent index, delivery mirror, API error dumps) #58248

Description

@Mythology1989

Summary

Hermes 0.18 persists conversation text on the delivered side (after transform_llm_output) in three places, none of which offer a redaction hook or a retention/TTL setting. For deployments that run a PII tokenization plugin (tokens restored only at delivery time), these surfaces are the only places where restored PII lands on disk:

  1. state/rich_sent_index.json (gateway/rich_sent_store.py): records the final text of every rich send, keyed by chat_id:message_id, to resolve replies (Telegram does not echo rich content in reply_to_message). Capped at 1000 entries / 2000 chars, but entries have no TTL. Note it is written even with platforms.telegram.extra.rich_messages: false via the pipe-table-primary auto-routing (adapter.py, _content_is_pipe_table_primary), so there is currently no config that fully disables the surface.
  2. Delivery mirror (gateway/mirror.pySessionDB.append_message): send_message mirrors the delivered text into the target session's transcript unconditionally on success; cron deliveries are gated by cron.mirror_delivery / per-job attach_to_session, but the send_message path has no equivalent gate.
  3. sessions/request_dump_*.json (agent/agent_runtime_helpers.dump_api_request_debug): on 4xx / retries-exhausted errors the FULL request body (system prompt + conversation) is dumped unconditionally. It already runs redact_sensitive_text for secrets — but there is no plugin hook to extend redaction, and no way to disable or age out the dumps.

Proposal (any subset would help)

  • A transform_persisted_text (or reuse of transform_llm_output-style) plugin hook applied before text is persisted by the rich-sent store, the delivery mirror, and the request dumps.
  • Config knobs: TTL / max-age for rich_sent_index.json entries and request_dump_*.json files (mirroring cron.output_retention), a toggle for the send_message mirror, and a toggle to opt out of the pipe-table rich auto-routing.

Environment

Hermes v2026.7.1 (0.18.0), gateway + Telegram adapter, PII tokenization via a pre_gateway_dispatch/transform_tool_result/transform_llm_output plugin.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/gatewayGateway runner, session dispatch, deliverycomp/pluginsPlugin system and bundled pluginssweeper:risk-session-stateSweeper risk: may lose/corrupt/mis-associate session or context statetype/featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions