Skip to content

recall fuses relevance and heat only — no trust or provenance dimension in WRRF, so a poisoned memory outranks a good one #368

Description

@cdeust

The retrieve-phase class in arXiv 2604.16548 is retrieval corruption: "RAG poisoning (malicious entries ranked highest by embedding similarity)" and "memory entries overriding explicit user instructions." Required defense: a "trust-aware or provenance-aware retrieval policy" with consensus validation. The survey is explicit that the read-side filter Cortex relies on is not enough: "Retrieval-time filtering alone is insufficient", and that existing defenses "have not been stress-tested by adaptive LLM-driven red-teaming."

Current state (verified 2026-08-06)

Per docs/mcp-tools.md, the read path fuses vector + FTS + trigram + heat + recency via the PL/pgSQL recall_memories() WRRF, then reranks with FlashRank (core/reranker.py), then filters with core/memory_rules.py and core/abstention_gate.py.

Every one of those signals is a relevance or salience signal. None is a trust signal:

  • core/thermodynamics.py heat rises with access — so a memory that keeps getting retrieved keeps ranking higher, which rewards a successful poisoning rather than penalising it.
  • core/reranker.py scores query-passage relevance; a well-crafted malicious passage is more relevant, not less.
  • core/abstention_gate.py asks "does this answer the query" — the poisoned entry does.
  • core/memory_rules.py hard/soft rules are user-authored and static; they cannot express "distrust anything whose provenance is a fetched web page."

So the ranking function has no term that could ever demote a memory for being untrustworthy, and the heat term actively amplifies whatever gets retrieved.

Ask

Add a trust/provenance term to retrieval, consuming the attribute #365 introduces at write time.

  1. A provenance-derived trust factor in the fusion, not only as a post-filter — the survey's point is that filtering after ranking is insufficient.
  2. Break the heat feedback loop for low-trust provenance: retrieval of an untrusted memory should not compound its future ranking.
  3. Red-team the ranker: adversarial passages crafted to win on similarity, per the survey's note that defenses are untested against adaptive attacks.

Acceptance criteria

  • A test seeds a high-similarity, low-trust memory alongside a lower-similarity, high-trust one and asserts the ordering the policy intends — on both backends (§12.3).
  • The trust term's weight is sourced (§8): a benchmark under benchmarks/ or a dated measurement, not a chosen constant. No invented constants.
  • Retrieval quality on the existing suites is measured before/after with paired arms at one commit; a relevance regression is stated and justified or the change does not ship.
  • An adversarial-passage fixture set is committed and exercised in CI.
  • Zero surviving non-equivalent mutants on changed files (§12).

Relates to #365, #366, #363, #360.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions