Found reviewing #303, in that PR's own test helper — but the pattern is broader than one test.
The ambiguity
references_cache/ holds 379 .md, 214 .txt and 78 .json files, and 63 references have more than one, almost always a .md alongside a .txt for the same PMID:
PMID_39085194 -> ['PMID_39085194.md', 'PMID_39085194.txt']
PMID_30870453 -> ['PMID_30870453.md', 'PMID_30870453.txt']
doi_10.1039_C3EE42189A -> ['doi_10.1039_C3EE42189A.json', 'doi_10.1039_C3EE42189A.txt']
The two are not interchangeable — per #265, a .md typically holds open-access full text while the .txt is often just the abstract. Any consumer that resolves a reference with glob(key + ".*") and takes the first match therefore reads a filesystem-order-dependent file.
Measured effect
Running the same snippet scan twice, preferring each extension:
| first match |
snippets located in cache |
.md |
4471 |
.txt |
4400 |
71 snippets are findable in one variant and not the other. Whether they get checked at all currently depends on directory iteration order, which is not guaranteed across machines or filesystems. The .json files are CrossRef metadata rather than prose, so treating them as text is wrong outright.
Why it did not show up as a failure
#303's gate flags 4 truncations under either preference (0 after the fixes land), so the ambiguity is currently invisible. It is latent: a future truncation in one of those 71 snippets would be caught or missed depending on the machine.
Fix
Resolve a reference to all its non-.json cache files and check each, rather than picking one. That is strictly more correct for substring search and removes the ordering dependence. Concatenating them is not equivalent for #303's purposes — its check inspects what follows the snippet, and joining two files would manufacture a continuation across the boundary.
Worth checking whether evidence_snippet_audit.py and the vendored reference validator resolve caches the same way; if they do, their RENDERING/MISMATCH/NOCONTENT buckets inherit the same ordering dependence.
Fixed for #303's gate in that PR.
Found reviewing #303, in that PR's own test helper — but the pattern is broader than one test.
The ambiguity
references_cache/holds 379.md, 214.txtand 78.jsonfiles, and 63 references have more than one, almost always a.mdalongside a.txtfor the same PMID:The two are not interchangeable — per #265, a
.mdtypically holds open-access full text while the.txtis often just the abstract. Any consumer that resolves a reference withglob(key + ".*")and takes the first match therefore reads a filesystem-order-dependent file.Measured effect
Running the same snippet scan twice, preferring each extension:
.md.txt71 snippets are findable in one variant and not the other. Whether they get checked at all currently depends on directory iteration order, which is not guaranteed across machines or filesystems. The
.jsonfiles are CrossRef metadata rather than prose, so treating them as text is wrong outright.Why it did not show up as a failure
#303's gate flags 4 truncations under either preference (0 after the fixes land), so the ambiguity is currently invisible. It is latent: a future truncation in one of those 71 snippets would be caught or missed depending on the machine.
Fix
Resolve a reference to all its non-
.jsoncache files and check each, rather than picking one. That is strictly more correct for substring search and removes the ordering dependence. Concatenating them is not equivalent for #303's purposes — its check inspects what follows the snippet, and joining two files would manufacture a continuation across the boundary.Worth checking whether
evidence_snippet_audit.pyand the vendored reference validator resolve caches the same way; if they do, their RENDERING/MISMATCH/NOCONTENT buckets inherit the same ordering dependence.Fixed for #303's gate in that PR.