feat(premise-check): verify attributions the USER supplies, not only CODEC's claims - #298
Merged
Conversation
…CODEC's claims
codec_claim_check is directional — it validates assistant → user. This closes
the other direction, and it exists because of a real miss on 2026-07-24.
The user's LinkedIn post credited a practice to a commenter:
"The most useful reply I got was from SOMEONE WHO ends each session by
listing which of his standing rules the model actually used…"
A correspondent then attributed it to the user:
"YOU MENTIONED YOU end each session by listing which standing rules
actually fired…"
Both texts were in context. Nothing flagged it; the answer engaged with the
premise and reinforced it. That is worse than invention — nobody made anything
up, a false premise simply propagated as fact through the answer, the user's
reply, and onward to a third party.
SCOPE, deliberately tiny. Only attributions with an artifact IN CONTEXT are
checkable: the source is right there, so a discrepancy is a fact about two
strings rather than a judgement. Premises about the world, the business, or the
user's history have nothing to check against — a general premise-checker would
relocate the fabrication one level up, with CODEC "correcting" the user from
nothing, which is strictly worse than the gap it closes. Not built.
TONE. The note asks, never corrects: all that is provable is that two passages
disagree, not which is right (the user may be paraphrasing; the commenter may
have adopted it since). A confident correction here would be the same overreach
this codebase refuses, aimed at the user instead.
Four independent conditions must all hold before it fires, because a false
positive tells the user they are wrong about their own life. Detection is pure
pattern-matching plus content-word containment — no LLM call, deterministic,
testable. Kill switch: PREMISE_CHECK_ENABLED=false.
Verified against the real incident (containment 0.62, caught) and 9 near-miss
false-positive shapes including the case where the practice genuinely IS the
user's. Wired into both chat paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
codec_claim_checkis directional — it validates assistant → user. This closes the other direction, and it exists because of a real miss.The incident
The user's LinkedIn post credited a practice to a commenter:
A correspondent then attributed it to the user:
Both texts were in context. Nothing flagged it — the answer engaged with the premise and reinforced it.
That is a worse failure mode than invention: nobody made anything up. A false premise simply propagated as fact through the answer, the user's reply, and onward to a third party. There is no moment where a guard watching output would see anything wrong.
Scope — deliberately tiny
Only attributions with an artifact in context are checkable: the source text is right there, so a discrepancy is a fact about two strings, not a judgement.
Premises about the world, the business, or the user's history have nothing to check against. A general premise-checker would relocate the fabrication one level up — CODEC confidently "correcting" the user from nothing. That is strictly worse than the gap it closes, so it is not built.
Tone — asks, never corrects
All that is provable is that two passages disagree, not which is right (the user may be paraphrasing; the commenter may have adopted it since). A confident correction here would be the same overreach this codebase refuses, pointed at the user.
Four conditions, all required
A false positive here tells the user they are wrong about their own life, so the bar is the highest in the codebase: a third-party attribution in context, a second-person attribution in context, both substantial enough to compare, and content-word containment ≥ 0.55. Pure pattern-matching — no LLM call, deterministic, testable. Kill switch
PREMISE_CHECK_ENABLED=false.Verified
Real incident: containment 0.62, caught, credited_to
someone.9 false-positive shapes all silent, including the one that matters most — where the practice genuinely is the user's own. Plus: unrelated you-statements, thin phrases, partial topic overlap, a colleague quoted on a different subject.
18 tests · 218 chat/claim/premise/standing tests pass · ruff clean.