fix: stop the cache-name loop, share the NCBITaxon adapter, write the protocol (#697, #700, #696, #704, #705)

[realmarcin]

Closes #696, #697, #700, #704, #705.

#697 — the loop that costs money

linkml-reference-validator normalises a reference id to DOI: and builds its
cache path from that, so every cache miss it fills writes DOI_*.md
(etl/reference_fetcher.py:204-225). Every reader here builds doi_… from the
doi: citation, so the file it just wrote is unreachable — and per
src/communitymech/paths.py an unreachable cache is not a skip, it sends the
fetcher back to the network. Miss, fetch, write an unreadable name, miss
again. 133 files accumulated that way before #690.

The dependency is not ours to change; what is ours is that its output leaves the
tree in a state the next run can use. scripts/normalize_cache_names.py now runs
after it. Option 1 from #697; option 3 (accept both casings on read) stays
rejected — it makes local and CI answers differ, which is #694's failure.

Details that matter:

• Renaming goes via a temporary name — DOI_x.md → doi_x.md is a no-op
  where case is ignored.
• Both casings existing as distinct files is a conflict, not a rename:
  reported and left alone, because choosing between two fetches of the same
  reference is not a script's call.
• Only the prefix is normalised. 10.1134/S0026261716060059 is a real DOI;
  lowercasing the whole filename would break resolution rather than fix it.
• The canonical prefix map lives in communitymech.paths, imported by both the
  renamer and the checker. A second copy is how 133 references_cache files are named DOI_* while every citation says doi:, so 23% of the cache is invisible on Linux #690's two halves got out of
  step.

The guard caught a recipe I missed. I wired validate-references and
validate-references-all; the test immediately failed on repair-references,
which fetches on a miss exactly as validate does — its --dry-run is about not
editing the record, not about not writing a cache.

It also pins the subtler half: the validator's exit code must survive the
rename. A recipe ending with a successful rename reports success whatever the
validation said, turning an evidence gate into a no-op. Verified both ways (clean
record → 0, planted fabrication → 1) and mutation-checked by removing the step.

#700 — the audit, and its honest result

Searching for the shape (a substring deciding a skip) across tests/,
scripts/ and src/ found three sites. Only one was the defect, and it was
already fixed:

site	verdict
test_record_roots_are_shared's _SHARED_MARKERS	the instance — fixed in #689
test_snippet_rendering_artefacts:103	the check's own logic; reports only snippets that do not match
test_gtdb_coherence_validator:644	a parse-saving shortcut that decided coverage by substring — removed

That last one would have silently skipped a curation_note written with any
other spacing. 328 files parse in under a second, which is not worth a blind
spot.

any(marker in body) in test_cache_is_source_text_not_notes is left alone: the
full-text markers are written into the cache by the fetchers, so presence is the
definition rather than an exemption — and it already recognises both forms
deliberately, which is the trap that twice destroyed real full text.

So the pattern was not widespread. The rule is the deliverable.

#696 — the protocol

CLAUDE.md gains "Proving a gate can fail": the mutation ritual has three
ways to lie and two produce a false green — a test certified as able to fail
when it cannot, which is precisely what the ritual exists to prevent. All three
were hit in one session: an unapplied mutation (twice), stale bytecode, and a
mutation this filesystem cannot observe. Plus the fourth point, about damage
rather than lying: back up by copy, because git checkout -- discards unrelated
uncommitted work in the same file.

The #700 rule lands beside it, since both are about evidence that looks like
evidence and is not: a substring may decide what a guard looks at, never that
something is fine.

#704 — one adapter, so availability is one question

ncbi_domain and shared_taxon_ids each built their own OAK adapter from
sqlite:obo:ncbitaxon, with the same except Exception: return None. The two
copies always agreed, so nothing ever looked wrong. What the duplication broke
was a measurement.

When s3.amazonaws.com/bbop-sqlite/ncbitaxon.db.gz began answering 403 —
verified from a developer machine on both HEAD and a ranged GET, so it is
upstream, not a runner — I gated the tests that fail when
ncbi_domain._adapter() returns None and shipped. CI failed on thirteen more,
because a probe pointed at one copy is blind to every caller of the other.

communitymech.ontology_adapters is now the one construction site and
ncbitaxon_available() the one question. Both validators alias _adapter to
it, which keeps monkeypatch.setattr(module, "_adapter", …) working for the
tests whose subject is the absent case.

Re-measured properly: the whole suite run against an unreachable ontology,
not one module. tests/test_ncbitaxon_adapter_is_shared.py holds it —
identity (is, not equivalence), one selector literal under src/, and a
subprocess run of the gated modules made hermetic by an empty PYSTOW_HOME
plus an unroutable proxy, so it needs no network and cannot pull 13 GB the day
the 403 lifts. GATED_MODULES is pinned rather than discovered by scanning for
the fixture name — a scan takes its coverage from the thing it checks — and it
caught two modules I had missed on its first run.

The skip says SKIPPED, NOT PASSED, matching what shared_taxon_ids already
prints to stderr. #708 tracks that two gates are consequently off until the
upstream artefact returns.

The review found a false green in this PR's own mutation check (#709)

test_the_check_can_actually_fail copied a module to tmp_path, removed a
fixture argument, and asserted a red. It got one — from fixture 'requires_ncbi_adapter' not found, because pytest loads a conftest from the
test file's own directory and the copy had none. An unmutated copy fails
identically (31 passed, 6 errors), so the test passed whatever the mutation
did.

That is #696's failure mode, committed one commit after writing #696's
protocol, inside the file whose subject is the same discipline. No run shows
it; green is what it produces either way. Fixed in fefa833: the conftest is
copied alongside, and the test grew a control arm — the unmutated copy must
be green through the identical harness before the mutated arm's red means
anything.

CLAUDE.md gains that as rule 5. Rules 1–3 interrogate the mutated run and
rule 4 the restore; none catches a red with a second sufficient cause.

#705 — an interrupted rename stranded the fetch

Also from reviewing this PR's diff. rename goes via a temporary name, and
Path.rename clobbers its destination silently — that destination being a
cache file an earlier run died holding. Worse, doi_x.md.casetmp is a name no
reader resolves and canonical_cache_name returns None for, so a later pass
stepped over it: the reference reads as a MISS, and a miss goes back to the
network. #697's loop, reached from the other side.

rename now refuses when a leftover exists; recover() finishes the
interrupted rename; main sweeps orphans first. Mutation-checked by removing
the guard from the executed source and asserting the leftover is clobbered.

Verification

just lint                     clean (black, ruff, mypy)
pytest tests/                 2856 passed, 17 skipped
pytest, ontology unreachable  2828 passed, 42 skipped   <- the CI condition
CI validate-strict            green: 2771 passed, 100 skipped

The two suite runs differ only in skips. The outage run is not a guess at
CI's environment: PYSTOW_HOME points at an empty directory and an unroutable
proxy blocks the re-download, which reproduces the exact Downloading ncbitaxon.db.gz: 0.00B symptom seen on main.

CI is green because the ontology is unavailable and the checks skipped
loudly — not because a cache warmed. From the run log:

Downloading ncbitaxon.db.gz: 0.00B [00:00, ?B/s]
[taxon-ids] NCBITaxon is unavailable, so the shared-id check (#292) was skipped, not passed.
[gtdb-domain] NCBITaxon is unavailable, so the prokaryote-only check (#365) was skipped, not passed.

One test skips on macOS and says so rather than passing vacuously — the conflict
case needs two files a case-insensitive filesystem cannot hold.

Known unrelated failure: test_no_snippet_stops_mid_word fails locally on
data/isolates/Methylobacterium_REE_Ewaste_Platform.yaml, driven by an untracked
local cache; it skips on a clean checkout.

Filed from the review

issue	in this PR?
#705 interrupted rename strands a fetch	fixed — new code, belongs here
#709 mutation check was a false green	fixed — new code, belongs here
#706 normaliser's exit code discarded, so a conflict cannot fail anything	filed — policy call; validate-references is not a CI gate
#707 OAK cache key never rotates, pinning one ontology snapshot	filed — pre-existing pattern; caching strategy is its own change
#708 the #292/#365 gates are off while bbop-sqlite 403s	filed — tracking, so turning them back on is an action someone takes