Skip to content

v1.108.272 — A column recorded on the wrong exit is not a measurement

Choose a tag to compare

@jgravelle jgravelle released this 12 Aug 13:24
· 43 commits to main since this release

#440, reported by @rknighton.

Both non-fusion search_symbols exits built a score-only ledger input and handed it to extract_ledger_features, which reads identity / identity_match. Neither key was present, so identity_hit was recorded as 0 on every row whatever the identity channel found — by construction rather than by measurement.

His reproduction runs one query down two exits and prints the rows side by side: the default path records identity_hit=0 on an exact symbol-name match, while search_symbols_fusion records 1 for the same query and the same top result.

The defect is less interesting than what hid it

The v1.108.187 fusion fix left a comment asserting the non-fusion paths already carried identity. That exoneration was false. They carried the value nowhere the reader looks — the lexical path folds it into the BM25 total inside _bm25_score, and debug=True nests it under score_breakdown. Near the reader is not where the reader looks, and a comment claiming coverage reads as coverage. That comment is corrected in place, because leaving it standing preserves the trap for the third instance.

What changed

Both exits now build their ledger input through _ledger_identity_rows.

Identity is recomputed rather than threaded out of scoring, deliberately: semantic_only skips the identity channel entirely, so reusing the scorer's value would keep recording a default dressed as a measurement — the very thing being fixed. It is a pure function of the symbol's name/id and the query, and only the top three rows are read.

The rows go to the ledger only, never to attach_confidence. compute_confidence sniffs the same key when no has_identity_match is passed and scores it 1.0 known-true / 0.7 unknown, so sharing one input would have moved the published confidence of every non-fusion search. Recording a column must not move a number callers already read.

The history is not repairable, and no heuristic was invented

Pre-fix rows always passed top1_score and only omitted the identity key, so they are indistinguishable from an honest post-fix 0. identity_label_is_trustworthy keeps returning True for them and now says why.

search_symbols is the highest-volume producer in the ledger, so the contaminated share is far larger than the fusion case that predicate was written for. Do not read "the fusion rows are handled" as "the identity_hit column is clean" — it is clean only for rows written from this release on. The recency window is the only remedy.

Two consumers were affected, both correct for new rows and both still contaminated for old ones:

  • analyze_perf.identity_hits undercounted by however many name matches those searches made.
  • regret's vocabulary-gap signal is the conjunction not identity_hit and semantic_used, and the semantic exit passes semantic_used=True literally — so both halves held by defect on every such row. Above the confidence floor and recurrence threshold it was reporting a vocabulary gap it had not tested, into user-visible suggest_corrections patches.

Verification

tests/test_v1_108_272.py (9), proven non-vacuous by reverting only the two call sites against the same tree: 1 fails pre-fix, and the 8 passing on both sides are the unit-level and no-heuristic controls.

Full suite 7709 passed / 7 skipped / 0 failed, and 7703 / 13 / 0 under a clean 3.13 environment; ruff check src/ clean; all 9 CI jobs green.