feat(loki): ground every answer in typed records instead of prose - #212
Merged
Conversation
Loki answered a five-part "plan my day" prompt with three fabricated
answers, then produced an unverified correction when challenged. Traced:
- "rotate the expired GROQ_API_KEY, it blocks truthseeker" — the key
returns HTTP 200. Both cited .env paths were invented.
- "Ilya Druzhnikov (UZH)" — the record has no org field and "UZH"
exists nowhere in the operator's data. It is the substring inside
dr-UZH-nikov: a keyword-match artifact narrated as an affiliation.
- "Elena Weber, Accelerator & Bridge Program Manager, University of
Liechtenstein alumna" — the record is a display name and a phone
number. No web search was performed.
Root cause is architectural, not prompt quality. buildLokiFleetContext
injected projects + RAG only; people, goals, habits, commitments and
events were never in context, though FleetCrown could already answer
every one of those questions exactly (getStuckGoals, getGoalsDueSoon,
listUpcomingCommitments, getEventsDueSoon have existed for months).
So a prompt demanding five items met a context supporting one, and the
model filled the rest — the ordinary failure mode of a small model
under format pressure, which is precisely who will be running this.
The fix makes unsupported claims hard to EXPRESS, not just discouraged:
agent/core/facts.ts records with a declared field set; unstored
fields render as an explicit `<not recorded>`,
turning absence into a stated negative
agent/core/contract.ts rules generated from THIS turn's facts —
enumerated citation ids, named concrete gaps,
a verbatim refusal phrase; plus Directive for
answers computed in SQL that the model may
only phrase, never re-derive
agent/core/verify.ts deterministic post-generation check. No extra
model call, so it runs on free-tier turns too.
Flags unresolvable citations and proper nouns
/ numbers / paths absent from the records.
agent/brief.ts wires the existing SQL into the assistant at
last; an empty result is reported as empty
agent/sources.ts people (new), projects, pgvector docs, and
OrangeCat demand → Facts
agent/context.ts assembly; contract first, records last
loki-core now verifies each answer and gives the model one repair pass
to DELETE unsupported claims. Surviving violations ride back as
`grounding` metadata rather than being swallowed: the original failure
was not that Loki was wrong, it was that wrong looked exactly like right.
scripts/test/agent-grounding.ts asserts all four fabrications above are
caught and that a clean grounded answer still passes. core/ is mirrored
to OrangeCat with a SHA-256 drift gate, pending @fleet/agent-core.
Removes loki-fleet-context/-index — superseded, and a second definition
of the fleet index would be the SSOT violation this replaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The verifier's original closed-world rule — every proper noun must be
attested in the records — is right for Loki and wrong for Cat, and
shipping it unchanged to both would have made Cat unusable.
Loki reports one operator's records. Any unattested name there is a
fabrication. Cat also answers general questions ("how do I get paid in
Switzerland"), where naming Twint, Lightning or PayPal is correct and
required. Flagging those would have produced constant false positives on
a live multi-tenant product, and a check that cries wolf gets switched
off within a week — at which point it protects nothing.
So `mode`:
closed-world every unattested proper noun is a violation (Loki)
entity-attribution only sentences naming one of the USER'S OWN records
are checked (Cat)
The narrower mode is genuinely weaker and that is a real trade, not a
loophole: Cat can still be wrong about the wider world. It can no longer
invent an employer for someone in your contacts — the failure that
actually destroys trust in a personal assistant.
Also adds buildAssistantRules() — the fact-id-free half of the contract,
for an assistant whose context is still prose. Weaker by construction
(nothing to cite), but it keeps the rules that stopped the worst failure:
never state an attribute a record does not carry, never imply research
you did not perform. A stepping stone to typed records, not a
destination.
Two new adversarial checks assert BOTH halves — general economic advice
passes in entity-attribution mode, an invented employer for a known
contact does not, and closed-world remains strictly stronger.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure this fixes
Loki answered a five-part "plan my day" prompt with three fabricated answers, then produced an unverified correction when challenged. Every claim traced:
GROQ_API_KEY, it blocks truthseeker".envpaths were invented.UZHappears nowhere in the operator's data — it is the substring inside drUZHnikov. Loki's own retraction ("an internal keyword search grouping error") was literally true.Root cause is architectural
buildLokiFleetContextinjected projects + RAG only. People, goals, habits, commitments and events were never in context — though FleetCrown could already answer every one of those questions exactly (getStuckGoals,getGoalsDueSoon,listUpcomingCommitments,getEventsDueSoonhave existed for months, unwired).So a prompt demanding five items met a context supporting one. The model filled the rest. That is the ordinary failure mode of a small model under format pressure — precisely who will be running this.
The fix: make unsupported claims hard to express
agent/core/facts.ts— records with a declared field set. Unstored fields render as an explicit<not recorded>, turning absence into a stated negative the model must actively contradict.agent/core/contract.ts— rules generated from this turn's facts: enumerated citation ids, named concrete gaps, a verbatim refusal phrase. PlusDirective, for answers computed in SQL that the model may only phrase.agent/core/verify.ts— deterministic post-generation check. No extra model call, so it runs on free-tier turns too — where fabrication is most likely and a frontier-cost verifier would be disabled.loki-corenow verifies and gives the model one repair pass to delete unsupported claims. Surviving violations ride back asgroundingmetadata rather than being swallowed: the original failure wasn't that Loki was wrong, it's that wrong looked exactly like right.Verification
scripts/test/agent-grounding.ts— 12 adversarial checks built from the real transcript. All four fabrications above are caught; a clean grounded answer still passes (the check is not a DoS on itself). 60/60 unit files pass, tsc clean, 0 lint errors.core/is mirrored byte-for-byte into OrangeCat with a SHA-256 drift gate in both repos (orangecat#feat/cat-grounding-harness). Exit is extraction to@fleet/agent-core.Removes
loki-fleet-context/-index— superseded; a second definition of the fleet index would be the SSOT violation this replaces.🤖 Generated with Claude Code