Found reviewing #298, where two snippets I wrote were clipped to near-uselessness and still passed every gate.
What happened
Snippet validation checks only that the text is a substring of the cached reference. It says nothing about whether the substring supports the claim. Two snippets in the draft were:
snippet: 3 Bacillus (Bl13
snippet: keystone taxa (Psedomonas, Duganella, Brevibacterium) interaction with C.
Both validated cleanly. The first is 16 characters and supports essentially nothing; the second stops mid-binomial at "C." and drops the strain it was meant to identify.
Two distinct causes
- Non-ASCII characters truncate snippets. The source writes isolate codes with U+2010 HYPHEN (
Bl13‐2C11), not ASCII -. Copying a fragment that stops before the hyphen is the path of least resistance, and nothing objects. Including the real U+2010 works fine — YAML and the validator both handle it — but you have to notice.
- Sentence splitting on "C. fusca". Naive sentence boundaries treat the abbreviated genus as a full stop, so a snippet extracted that way ends at
with C. and silently loses fusca CHK0059 within plant hosts were identified.
Both were fixed in #298 by using the full clauses.
Why a gate is plausible
There is a cheap signal here. Across the two records in #298 the shortest legitimate snippet is 35 characters, and the repo-wide distribution would show whether anything shorter is ever meaningful. Options, roughly increasing in cost:
- Warn below a length floor in
evidence_snippet_audit.py — crude, but would have caught the 16-character case.
- Flag snippets ending in a single-letter abbreviation (
with C., by S.), which is the sentence-splitter signature.
- Flag snippets that end immediately before a non-ASCII character in the cache, which is the U+2010 signature and is checkable because the cache is right there.
None of these can judge whether a snippet supports its explanation — that stays a curator's job. They only catch the mechanical truncation cases, which are the ones that slip through precisely because they look fine.
Worth checking first
Whether existing records already carry such snippets. If a length histogram over kb/communities/** shows a cluster of very short snippets, this is a real cleanup; if #298's drafts were an outlier, a lightweight warning is enough.
Found reviewing #298, where two snippets I wrote were clipped to near-uselessness and still passed every gate.
What happened
Snippet validation checks only that the text is a substring of the cached reference. It says nothing about whether the substring supports the claim. Two snippets in the draft were:
Both validated cleanly. The first is 16 characters and supports essentially nothing; the second stops mid-binomial at "C." and drops the strain it was meant to identify.
Two distinct causes
Bl13‐2C11), not ASCII-. Copying a fragment that stops before the hyphen is the path of least resistance, and nothing objects. Including the real U+2010 works fine — YAML and the validator both handle it — but you have to notice.with C.and silently losesfusca CHK0059 within plant hosts were identified.Both were fixed in #298 by using the full clauses.
Why a gate is plausible
There is a cheap signal here. Across the two records in #298 the shortest legitimate snippet is 35 characters, and the repo-wide distribution would show whether anything shorter is ever meaningful. Options, roughly increasing in cost:
evidence_snippet_audit.py— crude, but would have caught the 16-character case.with C.,by S.), which is the sentence-splitter signature.None of these can judge whether a snippet supports its
explanation— that stays a curator's job. They only catch the mechanical truncation cases, which are the ones that slip through precisely because they look fine.Worth checking first
Whether existing records already carry such snippets. If a length histogram over
kb/communities/**shows a cluster of very short snippets, this is a real cleanup; if #298's drafts were an outlier, a lightweight warning is enough.