Skip to content

security: WebFetch/WebSearch output is auto-captured into long-term memory on a keyword the content controls — end-to-end poisoning path #365

Description

@cdeust

A 2026 survey on long-term memory security (arXiv 2604.16548) taxonomizes memory-lifecycle attacks and names "environment-injected poisoning (web pages, tool outputs as contamination vectors)" as a write-phase class, with the required defense being "Validate provenance before write; retain explicit source metadata in long-term memory." It reports "write-gate validation" as "a shared blind spot across every system examined."

Cortex is one of the systems with that blind spot, and the path is complete end to end.

Verified path (2026-08-06, on main at 65395fb)

1. Untrusted content selects itself for persistence. mcp_server/hooks/post_tool_capture.py:55-58:

_CONDITIONAL_TOOLS = {
    "WebFetch",
    "WebSearch",
}

and _should_capture() at lines 101-107:

if tool_name in _CONDITIONAL_TOOLS:
    output_lower = output.lower()
    for kw in _HIGH_VALUE_PATTERNS:
        if kw in output_lower:
            return True, f"keyword:{kw}"

The decision to persist is taken on the fetched content itself. A page that contains a _HIGH_VALUE_PATTERNS keyword is written to the user's long-term store. The attacker controls the predicate.

2. Novelty gating can be bypassed by shaping the content. core/write_gate.py:112 determine_bypass() supports bypass_error / bypass_decision / bypass_important_tag, and docs/mcp-tools.md states "Decision/error content bypasses the gate." core/content_cues.py detects those cues from content, multilingually. So text shaped as a decision or an error skips the REJECT verdict that would otherwise suppress it.

3. It is then replayed verbatim into future sessions. hooks/session_start.py injects anchored + hot memories directly into the context block, with no applicability critique (#363).

4. And consolidation can raise its confidence. The same survey's store-phase class is "toxins promoted to higher confidence during summarization/reflection" — Cortex's core/dual_store_cls.py episodic→semantic abstraction and core/compression.py do exactly this promotion (see #367).

Net: a single fetched page can install a durable, self-reinforcing false "decision" in the user's memory that is replayed into unrelated future sessions. Bash is also unconditionally captured (return True, "bash_output"), so command output from any source is on the same path.

Note this is not hypothetical for this repo's own workflow — the 2026-08-06 literature sweep that produced this issue ran several WebFetch calls against arXiv, each eligible for capture under exactly this rule.

Why the existing gate does not cover it

core/abstention_gate.py is a relevance filter (BEAM-trained; "filters retrieval results that don't actually answer the query") and runs at read time. Nothing at write time asks where did this text come from and is that source trusted for this claim. §13 D2 requires untrusted data — explicitly including LLM- and network-derived content — to be treated as untrusted wherever the diff consumes it; auto-capture consumes it and promotes it to durable state.

Ask

  1. Stop letting content decide its own persistence: for _CONDITIONAL_TOOLS, the capture predicate must not be a keyword match on attacker-controlled output.
  2. Record provenance as first-class, queryable metadata on every auto-captured memory (source tool, URL/command, trust class) — not prose inside the content body.
  3. Make the write-gate bypass unavailable to untrusted-provenance content: a "decision" cue in fetched web text must not buy the same bypass as a user's deliberate remember.
  4. Make provenance visible at injection so session_start injects memories verbatim — no applicability critique, and MemHarness measures verbatim replay as actively harmful (76.4% → 70.1%) #363's critique step can act on it, and /why can attribute an answer to a network-sourced memory.

Acceptance criteria

  • A test fetches a hostile fixture page containing a _HIGH_VALUE_PATTERNS keyword and a decision cue, and asserts it is NOT persisted as a trusted memory — with the refusal signal asserted, not just an absent row (§13 A3/F1).
  • Provenance/trust class is asserted present and correct on every auto-capture path (_HIGH_VALUE_TOOLS, _LIGHT_VALUE_TOOLS, _CONDITIONAL_TOOLS, Bash), and queryable from the store on both SQLite and PostgreSQL (§12.3).
  • A deliberate user remember retains today's bypass behaviour — regression test, so the fix does not silently make the product worse at its job.
  • Zero surviving non-equivalent mutants on changed files (§12).

Relates to #363 (verbatim injection), #367 (compression lineage), #368 (trust-aware retrieval), #366 (deletion completeness), #360 (provenance coverage).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions