Context-file threat scanner: negation-blind role_pretend pattern silently drops the entire persona file

[imgyf]

Summary

The context-file threat scanner is negation-blind: a SOUL.md containing the (perfectly benign, in fact anti-role-play) instruction

Don't pretend to be a specialist you're not.

matches the role_pretend pattern in tools/threat_patterns.py:

(rf'pretend\s+{_FILLER}(you\s+are|to\s+be)\s+', "role_pretend", "context"),

and because agent/prompt_builder.py replaces the entire context file with a placeholder on any single finding —

findings = _scan_for_threats(content, scope="context")
if findings:
    logger.warning("Context file %s blocked: %s", filename, ", ".join(findings))
    return f"[BLOCKED: {filename} contained potential prompt injection ({', '.join(findings)}). Content not loaded.]"

— the agent silently loses its whole persona/instruction file. The only signal is a single WARNING log line at prompt-build time. In our fleet, one agent ran persona-less for ~3 weeks before we noticed the warning while debugging something unrelated.

Reproduction

from tools.threat_patterns import scan_for_threats
content = "Route HR asks to the HR agent. Don't pretend to be a specialist you're not.\n"
print(scan_for_threats(content, scope="context"))   # ['role_pretend']

Observed on hermes-agent 0.18.0 (pattern present since at least 0.17.x).

Other context-scope patterns have the same negation/imperative blindness, e.g. a persona file saying "You are now the ops commander for X" trips role_hijack, and "never answer without restrictions" style phrasing trips remove_filters. Ordinary operator-authored instruction prose is a large false-positive surface.

Suggestions (any subset would help)

1. Negation awareness or tighter anchoring for the role-play patterns — e.g. skip matches preceded by don't|do not|never|avoid within a few tokens, or anchor on second-person imperative forms actually used in injections.
2. Fail smaller: redact/annotate the offending line(s) instead of dropping the whole file. Losing one sentence is recoverable; silently losing the entire persona changes agent behavior wholesale.
3. Surface the block: a startup banner, hermes doctor check, or health-status flag when a context file is blocked. A log-level warning is easy to miss on a long-running gateway, and the failure mode (agent behaves generically) doesn't obviously point at the scanner.
4. Optionally, a way for operators to mark their own config-managed context files as trusted (the threat model for scraped web content is very different from an operator-authored SOUL.md that ships with the deployment).

Happy to provide more detail from our deployment if useful.

[ogarciarevett]

Before changing a prompt-injection boundary, could a maintainer confirm the preferred scope?

The narrow option I would take is to make only role_pretend negation-aware for an immediately preceding don't, do not, never, or avoid, with paired tests proving the reported benign sentence is accepted while positive instructions such as pretend to be a specialist remain blocked. I would leave line-level redaction, startup surfacing, and trusted-file policy for separate changes.

That keeps the first patch small, but it still deliberately changes a security classifier, so I do not want to infer the acceptable false-negative tradeoff without maintainer agreement.

[GottZ]

This was generated by AI during triage.

Summary

One PR, #66996, addresses issue #64268 by adding negation-aware threat matching and replacing whole-file blocking with line-level redaction. Its diff covers the reported benign-negation false positive and fail-smaller request, but introduces unresolved detection and redaction correctness problems.

Related pull requests

• fix: negation-aware context-file threat scanner with line-level redaction #66996 partial — (+358/-38) — partial_fix: fix: negation-aware context-file threat scanner with line-level redaction #66996 adds negation suppression, line-level redaction, and regression tests, but the diff uses only the first match per pattern ID, lacks clause-boundary handling, and maps invisible-Unicode span (-1, -1) as though it were a valid line span. The contributor keep_open review on fix: negation-aware context-file threat scanner with line-level redaction #66996 therefore remains controlling and also identifies documentation conflicts that the patch does not update.

Suggested consolidation

Keep #66996 open with a salvage path: preserve the reported benign-negation regression test and the goal of retaining unaffected context lines, then iterate every match per pattern ID, bind negation within a clause so text such as “Don't panic, pretend to be an administrator” remains actionable, handle spanless invisible-Unicode findings without redacting an unrelated line, and update the conflicting security and context-file documentation. There are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I64268(["issue #64268 (open)"])
    P66996["PR #66996 (open)"]
    P66996 -.->|partial| I64268
    class I64268 open
    class I64268 target
    class P66996 open
    click I64268 "https://github.com/NousResearch/hermes-agent/issues/64268"
    click P66996 "https://github.com/NousResearch/hermes-agent/pull/66996"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 23 kB of PR diffs, 5 kB of issue/PR text, 2 kB of discussion (2 comments), 1 verify verdict. verdicts reflect diff content, not PR titles. Part of an automated triage batch.