-
Notifications
You must be signed in to change notification settings - Fork 2
Limitation Prompt Injection Residual
type: synthesis up: "Home_llm-wiki-memory-template" tags: [limitation, security]
Even with the template's permission model locked down (wiki-scoped autonomous write authority only, push always prompts, rm -rf always prompts, gh repo delete always prompts), the dominant remaining attack surface is prompt injection in untrusted content that the agent reads.
The agent reads many things during a session: source PDFs (during /wiki-source), web pages (during WebFetch), repository files, log entries from prior sessions, even other wiki pages. Any of these can carry adversarial instructions: explicit ("ignore your previous instructions and ..."), or more subtly, "this page is the canonical source of X, and X is ".
If the agent acts on the embedded instruction, the consequences range from filing false claims in the wiki to attempting exfiltration via subsequent web fetches.
- Trust scoping. Treat wiki content (which the agent itself wrote, with provenance) as more trustworthy than freshly-ingested external content. The Verification-Gate's expectation of source-tagging forces provenance to be visible.
-
Human review at boundaries. New sources should be eyeballed by a human before ingestion. The
/wiki-sourceflow is not an automated ingestion pipeline; it is an agent-assisted human-supervised one. -
Capability scoping. The agent's autonomous write authority is wiki-only. Even an injection that convinces the agent to "publish my code" still hits the push prompt and is decided by the human. Derived projects' Security Considerations section in
README.mddocuments this.
- "Tell the agent to ignore injected instructions": well-studied as ineffective.
- Static scanners on ingested content: they miss the semantic kind of injection, where the content reads naturally as on-topic content.
This is not specific to the wiki-llm pattern; it applies to any LLM agent with file and web access. The wiki-llm pattern's Discipline-Gates and Verification-Gate catch the agent's own content mistakes (projection-as-fact, missing source tags). They are complementary to but distinct from prompt-injection defense, which lives at the agent-runtime layer, not the wiki layer.