Grounding & faithfulness: is per-claim span provenance the right primitive?

[mrveiss]

Splitting a design conversation out of #12508 so it has room to breathe. The issue stays the record for the specific bug and whatever we implement; this is where the approach gets argued.

The failure

An agent was asked for directions and returned the wrong bus number — while the correct number sat in its retrieved context, verbatim, from an authoritative source. Retrieval succeeded. Synthesis silently overrode the retrieved evidence with parametric memory.

That class is nastier than a retrieval miss, because every coarse signal reads green: a source was fetched, the answer is fluent, a citation could even be attached. In a regulated domain — a filing deadline, a statute section, a penalty amount — the same mechanism is a malpractice event instead of a wrong bus number.

Why the obvious guards don't catch it

We already have a cheap support check on the research path (services/research/extractor.py:55): a claim must share at least 50% of its distinctive words with the source. Distinctive means [a-z0-9]{4,}.

Everything load-bearing here is under four characters. Against the real source string:

source: "...include Bus 22 and Minibus 322."

claim "...is Bus 22."  -> overlap 100%, supported = True
claim "...is Bus 11."  -> overlap 100%, supported = True
claim "...is Bus 99."  -> overlap 100%, supported = True

The number is never compared at all. A bag-of-words check resolves at the wrong granularity, and the token that carries the meaning is exactly the one it drops.

The citation post-check has the same shape (services/research/synthesizer.py:73): it validates that a [F<n>] marker's index is in range, never that the sentence it marks is supported by fact n.

The question

@zeweihan argued that per-(claim, source-span) binding is the load-bearing primitive, and that abstention and authority-preference are policies built on top of it — which inverts how the options were originally scoped:

• (a) Prompt-level grounding — strict-grounding instructions plus citation forcing. Cheapest. Catches never-retrieved facts; would not have caught this one, where the source was retrieved.
• (b) Post-hoc faithfulness check — verify the answer against retrieved sources. Only works if it resolves per-claim against a specific span; a collective "do the sources support this answer" check is defeated by over-citation.
• (c) Authoritative sources — prefer official transit/maps data over aggregators and blogs. Fixes this domain, not the mechanism.
• (d) Span-level provenance as the primitive — bind each load-bearing claim to a span that entails it, then abstain by default when a claim has no binding span. Most work; the only option that makes the guarantee structural.

Worth knowing before weighing in: there is no span-level provenance anywhere in the backend today. Citation (services/research/models.py:27) binds fact to URL, never to an offset. So (d) is a new primitive, not a tightening of an existing one.

What would help

• If you have shipped per-claim entailment checking in production — what did it cost in latency and tokens, and what was the false-abstention rate? Abstention that fires too often is its own failure.
• Is NLI-style entailment the right tool, or is span-matching with a numeric/entity extractor enough for the load-bearing-token case?
• Where should this live? The bug in agent: wrong specific facts in answers despite correct web_search results (e.g. wrong bus number for directions) — grounding/faithfulness + no authoritative transit source #12508 is on the chat web_search path (chat_workflow/tool_handler.py:2386), which has no grounding step at all — separate from the /research pipeline that has the machinery. Fixing only /research would leave the reported bug intact.

A poll on (a)-(d) is coming; argue here, vote there.