fix: three gates that could not see what they checked (#686, #689, #690)

[realmarcin]

Three gates that could not see what they claimed to check. Each commit is
independent and separately mutation-checked.

#690 — 133 caches named so only macOS could find them

references_cache/ carried two conventions for one prefix — 133 DOI_* and 79
doi_* — while every reference writes lowercase doi: (1096 occurrences, zero
uppercase). The stem convention shared by this repo's scripts and by the
upstream validator produces doi_…, so the DOI_* half was findable only on a
case-insensitive filesystem. macOS is; Linux is not.

Not tidiness: linkml_reference_validator's fetcher builds the same stem and
calls .exists() (etl/reference_fetcher.py:204-225), so on Linux those were
cache misses — and per src/communitymech/paths.py a miss sends the fetcher
to the network. 117 of 514 distinct references (23%) were in that state.

All 133 renamed. git mv DOI_x.md doi_x.md is a no-op under
core.ignorecase=true, so each went via a temporary name; verified afterwards
that 0 tracked DOI_* remain, 212 doi_* exist (79 + 133), no .casetmp was
left behind, and no stem collides in both cases.

tests/test_reference_cache_names_are_case_exact.py stops it returning. It uses
os.listdir rather than Path.is_file(), because is_file() on a
case-insensitive filesystem answers this exact question with a "yes" Linux would
not give — the check has to be able to fail on the machine running it.

#686 — the KGX gate ran on nothing, hiding three real mismatches

kgx_nodes globs output/kgx/nodes.tsv; output/kgx/ is gitignored with zero
tracked files, and no job built the export before just validate-products. So
the glob matched nothing, the target skipped, and the gate printed "All id↔label
pairs correspond" having read nothing.

Three genuine MISMATCHes were behind that skip:

CHEBI:49591 label='europium(3+) cation'  canonical='europium(3+)'
CHEBI:49650 label='holmium(3+) cation'   canonical='holmium(3+)'
CHEBI:49746 label='lutetium(3+) cation'  canonical='lutetium(3+)'

They are the same + cation typo already waived once for CHEBI:16793
("mercury(2+) cation") in a hand-written metal→CHEBI map. The waiver called it
an exporter-side design choice. It was a typo, and it had been covering for
three more of its own kind. All four are fixed at the source and the waiver is
deleted rather than multiplied — mercury now passes as OK_CANONICAL, which
also confirms the canonical form the waiver asserted. Exceptions 13 → 12,
errors 0.

The workflow now runs just kgx-export before both report-label-drift and
validate-products, and the target is required: true so a missing artifact is
a MISSING_GLOB error rather than a skip.
tests/test_generated_gate_inputs_are_built_in_ci.py holds both halves
together — either alone restores the hole.

output/kgx/** leaves the paths: filter: a gitignored directory can never put
a file in a PR, so it advertised a trigger that could not fire.
test_ci_triggers_cover_what_the_gates_read.py was requiring that — right for
committed data, meaningless for generated data — so generated roots are exempted
there and carry the build-before-read obligation in the new module instead.

#689 — a test's record-root scope is now a decision

test_no_constructor_hardcodes_the_old_root scans src/communitymech/**, but
many gates here are tests, so one globbing kb/communities alone was
invisible to it. That is how the #529 discriminator swept one root for months.

Only modules that sweep are classified — naming one record as a fixture is not
a scope decision. 14 sweep. The split is measured, not assumed: data/isolates
holds 4 records with 66 snippets, 3 ecological_interactions, 3
gtdb_classification blocks, 0 cultivation_setup, 0 go_terms, and none is
rendered into docs/ (324 pages for 324 community records).

• _COMMUNITY_ONLY (3) — scoped by argument or by following the rendered pages.
• _OWED_BOTH_ROOTS (11) — each with what is being missed. Two miss nothing
  today only because isolates carry no cultivation_setup or go_terms yet, and
  would not notice the first that does. test_network_auditor passes
  communities_dir= explicitly, overriding the auditor's shared default — the
  shape data/isolates/** now triggers validate-strict, but that job does not validate isolates #350 fixed in the auditor and left in its test.

Emptying _OWED_BOTH_ROOTS is follow-up; this makes the 12th addition a choice
someone defends.

Verification

just lint · validate-products (0 errors, 12 exceptions)     PASS
pytest tests/                     2818 passed, 16 skipped

Every new assertion mutation-checked, each mutation confirmed applied before
a red was trusted: rename a cache back to DOI_*; drop required: true; move
the build after the gate; delete the build step; remove an _OWED entry; plant
a nonexistent module.

Known unrelated failure: test_no_snippet_stops_mid_word fails locally on
data/isolates/Methylobacterium_REE_Ewaste_Platform.yaml, driven by an
untracked local cache. Untouched here; skips on a clean checkout.

---

Adversarial review (commits 4–5)

The rename was only half the fix

scripts/cache_fulltext.py::_doi_cache_path preferred DOI_ and fell back
to doi_ last, so a DOI with no cache yet was created uppercase —
unreachable the moment anything read it back. That is how the 133 accumulated,
and renaming them without this would have refilled the set on the next fetch.
Canonical is now lowercase; uppercase is still read, so an old local file is
found rather than silently re-fetched, but it is never the name a new file gets.

A false red, from stale bytecode — #693

The test pinning that fix first loaded the script with
spec_from_file_location. scripts/ is not a package, so that writes
scripts/__pycache__, which Python validates on (mtime, size) — and
doi_→DOI_ changes neither. A mutation and its restore within the same second
were indistinguishable to the loader, which served a stale module and
produced a failure with correct source on disk.

The direction that bit was harmless. The same mechanism produces the opposite: a
mutation check going green from the stale mutated module, certifying a test
that cannot fail. The test now compiles the source text on every run and creates
no __pycache__. Re-verified: the mutation fails, and the restore returns to
green immediately rather than one run later. Filed as #693 — 36 test
modules use that loading pattern.

The legacy-read assertion was also platform-dependent (path equality holds on
Linux, not on macOS where the lowercase candidate already .exists()). It now
asserts the returned path exists and holds the right content.

The rename exposed a gate CI could never run — #694

CI failed test_no_snippet_stops_mid_word on this branch, caused by this
branch. _cached globbed {key}* then {key.upper()}*; the second could never
match, because it upper-cases the whole stem (doi_10.1128_aem… →
DOI_10.1128_AEM…) when only the prefix was ever uppercase. So on Linux the
reference resolved to nothing and the record was skipped, while on macOS the
filesystem found it and the check ran — the gate reported on a different
corpus depending on where it ran.

Six findings in Tinto_River_Iron_Cycling_Community.yaml sat on the wrong side
of that split. They are cache artefacts, not bad quotes: the abstract's italic
taxon names lost their surrounding spaces when fetched, so "...related to Leptospirillum spp., Acidithiobacillus..." is cached as "...related toLeptospirillumspp.,Acidithiobacillus...". The snippet ends at a real word
boundary in the article; only the cache disagrees. Waived in
_CACHE_RUN_TOGETHER with the reasoning recorded, mutation-checked (removing
the entry flags all six again). The dead key.upper() pattern is deleted rather
than repaired.

Also extended: test_generated_gate_inputs_are_built_in_ci named one workflow,
so a second workflow running the gate without building would reopen #686 with
every assertion still green. It now checks every workflow.

Final CI

label-correspondence · lint · validate-strict · vendored-sync · python-compatibility

all SUCCESS. label-correspondence is the meaningful one: it is the first run in
which that job actually built and read the KGX export.