The Trust Layer

The Trust Layer

Semantic similarity answers "which memory looks most like the query". It cannot answer "should this memory still be believed". The trust layer is that second judgment.

It runs as pure post-processing over a retrieval result — no database access, no clock reads, no I/O. Everything it needs (the supersession map, the calibration, the current time) is passed in. Source: recall/trust.py.

What comes back

Every hit is annotated with four things, not one:

verdict	Whether the memory can be relied on, and if not, why
confidence	Calibrated, on a scale where the midpoint sits exactly on the abstention boundary
provenance	Source, file, chunk ordinal, and when it entered the index
validity	The window it is valid in, and its successor if it has been superseded

And the result as a whole carries an abstained flag with a human-readable reason, a calibrated flag, calibration and generation identity where available, a corpus gap_warning, and a staleness report on the index itself.

Only ok hits should be acted on. That is the entire contract, and it is why the return type is a judgment rather than a ranked list.

The verdicts

Verdict	Meaning
ok	Current, in-window, above the confidence boundary. Trust it.
superseded	A later memory declared that it replaces this one. validity.superseded_by names the successor.
expired	Past its authored valid_until.
not_yet_valid	Before its authored valid_from.
low_confidence	Nothing wrong with it — it simply did not clear the calibrated threshold.
invalid_metadata	Its validity dates could not be parsed. Fails closed.
ambiguous_supersession	A supersession edge points at this memory's basename, but more than one document in the corpus carries that basename. Fails closed.
not_entailed	(Opt-in entailment stage) Semantically close, but does not actually answer the query.

Precedence when several apply: invalid_metadata > superseded > expired / not_yet_valid

low_confidence > ok.

Two of these are worth dwelling on, because they are where the design's principle shows.

invalid_metadata fails closed per hit. A memory whose validity window cannot be parsed is not served as healthy — but nor does it crash the search that retrieved it. One bad row must not take down every query that touches it, and it must not read as trustworthy either. (It is reachable only by writing directly to the store; the indexer refuses malformed dates up front.)

ambiguous_supersession refuses rather than guesses. If an edge names notes.md and two directories each contain a notes.md, which document the author meant is genuinely unanswerable. Serving the hit as ok — quietly dropping the unresolvable edge — would be exactly the silent wrong answer this layer exists to prevent. So it fails closed and the abstention reason tells you to disambiguate the corpus.

Supersession

Authored, not inferred

Supersession is a relation between two memories. Both look valid in isolation, so it cannot be recovered from the documents at read time. RE-call requires the author of the replacement to declare it, in frontmatter:

---
supersedes: rate_limits_v1.md
valid_until: 2026-12-31
---

That is the whole mechanism. A claim honoured as written is safe; a claim guessed at is not.

Why not just trust the newest hit

Because it was tested, steelmanned, and it does not work. Recency and authority are different properties: a memo can be newer than a memory it contradicts without being the one that replaced it, and "most recent relevant document" keeps selecting the stale memory at a rate that makes it useless as a guard. The numbers are in FINDINGS; the design argument is in The-Problem.

Chain resolution

Edges form a graph, and retrieval follows it to the terminal successor — if a is superseded by b and b by c, a hit on a reports c. Two degenerate cases are handled explicitly rather than left to crash:

• A cycle (a → b → a) stops on the first revisit rather than looping forever.
• A self-claim (a document declaring it supersedes itself — an authoring slip) is ignored, because a document cannot supersede itself.

Reference formats

A real corpus taught this one. On the reference corpus, every declared supersession edge failed to resolve — not because the targets were missing, but because of how the references were written: some as wikilinks ([[name]]), some without the .md extension, while the matcher compared full basenames. The linter dutifully reported "does not exist in the corpus", which was actively misleading.

The conclusion drawn was that a convention the corpus's own author cannot follow is a defect in the convention, not in the author. So name, name.md, [name] and [[name]] now all resolve to the same document — brackets stripped, extension stripped, compared on the stem. Directory prefixes are dropped too.

Ambiguity handling did not get the same treatment: two files sharing a stem are still refused rather than guessed at. Tolerance about formatting is safe; tolerance about identity is not.

Successor promotion

A subtle ordering rule that matters more than it looks. When a superseded hit scored above the threshold — meaning it would have been the confident answer — its retrieved successor is promoted from low_confidence to ok, even if the successor's own wording scores lower.

The reasoning: the explicit supersession edge transfers the topical relevance the stale memory just proved. The stale memo demonstrated the query is about this subject; its replacement is by declaration the current statement on that subject. Without this rule, a query phrased in the old memo's vocabulary would demote the stale hit correctly and then abstain — technically honest, but it would be withholding an answer the corpus actually contains.

Hits are then reordered valid-first, so the successor outranks the memory it replaced. This is the ordering decision the whole project turns on.

The residual failure mode, and why it ships as a reviewing aid

An author can forget to declare the edge. That is a much better place to fail than at read time, because the omission is lintable — which is what recall lint, recall fix and recall check address (CLI-Reference).

But the honest finding is that it could not be automated. On the reference corpus, dozens of memos described a closure in prose while declaring nothing in frontmatter. The extractor built to recover those edges proposed almost nothing, and on review every survivor was wrong for a different reason: one was reported speech, some superseded a claim or a scope inside their target rather than the target itself, and one was hedged — its author, asked directly, said the memo augmented rather than replaced.

Narrating versus declaring, part versus whole, augmenting versus replacing: invisible to a pattern, obvious to the person writing. So the tool ships as a reviewing aid, and recall check moves the question to write time, where the one who knows is still in the room.

Note the deliberate inversion between the two: fix refuses everything it cannot prove, because it writes unattended. check surfaces every candidate it can find, because a human is right there to pick one. Same extraction, opposite disposition — and the reason is who is in the room.

Validity windows

valid_from and valid_until are ISO dates in frontmatter, interpreted in UTC. valid_from starts at the beginning of its day; valid_until runs to the end of its day, inclusive — so a memo valid until a given date is valid for all of that date, which is what an author writing the date means.

Source: recall/frontmatter.py.

Two robustness details, both from real files:

• A UTF-8 BOM before the opening fence is tolerated. Windows editors add one, and a BOM that silently disabled frontmatter would mean validity metadata lost without any signal.
• Quoted values are unwrapped. Writing supersedes: "v1.md" out of YAML habit must match the unquoted filename, not silently never apply.

Both are the same class of bug: a guard that fails silently is worse than one that fails loudly, because nothing tells you the protection is gone.

Calibration and abstention

Why the threshold is calibrated per embedder

A fixed cosine threshold does not transfer across embedding models. Each model places its similarity scores in a different regime, so a threshold tuned on one is arbitrary on another. This was measured rather than assumed → FINDINGS §2.

recall calibrate fits a threshold against a labelled set of answerable and unanswerable queries and writes it to a file. Every search then maps raw cosine to a calibrated confidence through a monotone logistic curve centred on that boundary — so 0.5 is exactly the decision point, by construction.

That confidence is a calibrated ranking confidence, not a posterior probability. The calibration sets are small and the docstring says so; treating it as a probability would be reading more into it than the data supports.

How the threshold is placed

In the middle of the observed gap between the two score distributions — bisecting the answerable floor and the unanswerable ceiling, each taken as a quantile rather than an extreme so that a single outlier cannot define the boundary.

The previous rule minimised misclassification on the samples it was given. That sounds principled and is not: the cheapest way to keep every answerable sample above the boundary is to put the boundary exactly on the lowest one, which has three consequences that were all measured —

• no margin on the answerable side, so any real answer weaker than the weakest calibration sample abstains;
• one sample decides everything, because the answerable distribution has a long lower tail and the boundary sat at the bottom of it;
• it inherited ANN noise — HNSW index builds are nondeterministic, so the identity of the worst sample changed on every rebuild and the operating point moved with it.

The replacement was validated by fitting on half the queries and scoring on the other half, across multiple index rebuilds. It trades a small increase in wrongly-abstained answerable queries for a large cut in confidently-answered unanswerable ones. Going further was tested and rejected as a bad trade. Figures → FINDINGS.

⚠️ Outlier robustness needs samples. The floor is a low quantile, and a small tail is not identifiable from a handful of points — below roughly twenty answerable samples it collapses onto the minimum and one bad retrieval moves the boundary again. Bisecting the gap still adds margin at any size; a small calibration set buys margin, not stability. Calibrate against a few hundred labelled queries if the threshold matters to you.

Loading is defensive on purpose

A legacy calibration file is ignored — falling back to the uncalibrated default, with calibrated=False flagged in every result — when it is absent, unreadable, malformed, fitted for a different embedder, or carries out-of-range values.

Each of those is a real failure mode: a threshold fitted in another model's cosine regime must never be applied, and a corrupt file must never be able to silently disable abstention (a NaN threshold compares false against everything) or crash every search (a zero or negative scale). The flag matters as much as the fallback — a caller can tell that the guard is running on defaults.

In v1 generation serving, calibration is resolved from PostgreSQL by tenant and active generation. Legacy local files are import-only evidence and are not automatically selected by the MCP server. Production strict trust refuses when the active generation, lineage or certified calibration cannot be established.

Certification

Calibration now records whether the answerable and unanswerable classes were actually separable. Calibration.certified is false when the classes overlap or either class is too small, and None for older artifacts where the question cannot be answered. Certification changes no runtime threshold by itself; it is a gate and a warning surface, not a retune. That matters because moving the boundary during certification would mix diagnosis with behavior.

When it abstains

abstained is true when no hit earned verdict ok. The reason is specific, not generic: whether the best candidate was superseded (and by what), outside its validity window, carrying malformed metadata, ambiguously superseded, or simply below the confidence threshold — the last being the probable-corpus-gap case.

A separate, cheaper signal runs alongside: gap_warning fires when every dense candidate scores below the threshold. It is computed from dense cosines only, never the fused ranks, so a purely lexical match still reports a gap. That is deliberate — a keyword hit is not evidence of semantic relevance, and letting the sparse leg suppress the gap warning would quietly reintroduce confident answers over an empty corpus.

The near-miss, and the entailment stage

The threshold catches far gaps: queries whose best match is semantically distant. It cannot catch the near-miss — a memory adjacent to the query that does not answer it. Its similarity clears any threshold by construction, which is a stronger statement than "the threshold is badly tuned". No threshold works here.

The abstention signal for that class cannot come from the retriever's own score. It needs a separate judgment that the memory actually entails an answer. Proximity is a candidate; entailment is the evidence.

So apply_entailment is an opt-in stage that judges only verdict-ok hits — judging an already-demoted hit would waste a model call and could resurrect a superseded memory — demoting those that do not entail an answer to not_entailed, and abstaining when none remain.

It is off by default, and the honest reason is in the measurements: the judge cuts near-miss false-confidence substantially, but degrades far-gap detection. The two stages stack; neither replaces the other. Enabling it costs one judge pass per ok hit. Full study → docs/ENTAILMENT_SUPERSESSION_STUDY.md.

Staleness

Independent of any individual hit: the result reports how old the newest indexed content is, and flags the whole result stale past a maximum age. An index that stopped updating returns confident, well-formed, quietly obsolete answers — and nothing in the hits themselves would show it.

Strict trust and degraded development mode

Strict mode is the default. When the trust gate cannot certify an answer because the index is not ready, lineage mismatches, calibration is missing or uncertified, calibration is stale, or a needed dependency is unavailable, the production surface raises a refusal rather than returning an empty result. Empty means "the gate ran and found nothing trustworthy"; refusal means "the gate could not run." Collapsing those two would let an agent treat an outage as no prior memory.

RECALL_TRUST_MODE=development exists for local ad hoc corpora. It can return degraded results, but those hits are unverified context and must not be read as a trust-layer claim.

---

Next: Retrieval-Pipeline for what happens before the trust layer · Python-API-and-MCP for how to branch on these fields · CLI-Reference for calibrate, lint and check.