fix(convex): stop agents echoing <untrusted_source> tags into replies#2149
Merged
Conversation
The TRUST RULES system prompt told the model to 'quote the URL attribute of the enclosing tag' when citing, but never said the tags themselves are internal markers. Models read that as a citation format and reproduced the whole <untrusted_source ...>...</untrusted_source> wrapper in user-facing replies (intermittently, depending on how they chose to cite). Add an explicit rule to system.untrusted_content (and the researcher agent's inline copy) forbidding reproduction of the tags and pointing citations at normal markdown links. Update the cache-critical prompt snapshot accordingly.
651ecc4 to
f2ff4ff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Agents occasionally reproduced raw
<untrusted_source ...>...</untrusted_source>wrapper tags in their user-facing replies — sometimes once, sometimes around every fourth sentence. A teammate flagged it in a live chat (the model wrapped each cited fact in the literal tag).Root cause
This is a prompt-design gap, not model misbehavior:
<untrusted_source>tags viawrapUntrusted()inlib/untrusted_content.ts. That's a deliberate prompt-injection defense — the tags tell the model "this is data, not instructions."system.untrusted_contentTRUST RULES prompt only said "quote the URL attribute of the enclosing tag" and never stated that the tags are internal markers that must not appear in output.Fix
system.untrusted_content: replace the ambiguous line with two explicit rules — cite via theurlattribute as a normal markdown link, and never reproduce the<untrusted_source>opening/closing tags.researcher.json: same correction in the agent's inline TRUST RULES copy (it repeats the instruction and hits Tavily every turn, so it's the most visible offender).render.test.ts.snap: update the cache-critical prompt snapshot to match.Scope of effect
The shared
UNTRUSTED_CONTENT_SYSTEM_PROMPTcovers every agent whose tools return external content, so fixing it in one place (plus the researcher's inline copy) cleans it up everywhere.Tests
render.test.ts— 14 passed (snapshot matches)example-agents-normalized.test.ts— 2 passed (researcher.json still valid/normalizes)Not in scope
The separate question of why Tavily ran without being statically bound (composer capability-pill → per-turn dynamic binding via
mergeCapabilityBindings) is untouched — can follow up separately if desired.