validate-strict is red on main: it downloads the NCBITaxon DB every run, with no cache, and the download now returns 0 bytes

[realmarcin]

Summary

validate-strict is failing on main (run for 22673ba, 2026-08-29T02:03)
for an environmental reason, not a code change. The NCBITaxon SQLite database
download returns zero bytes on the runner:

Downloading ncbitaxon.db.gz: 0.00B [00:00, ?B/s]

so communitymech.validators.ncbi_domain._adapter() returns None, domain_of
returns None for everything, and the assertions that depend on it fail:

tests/test_ncbi_domain_scope.py:63    assert False is True
tests/test_ncbi_domain_scope.py:115   NO_GTDB_EQUIVALENT on an in-scope taxon
tests/test_ncbi_domain_scope.py:138   assert None == 'NCBITaxon:2'
tests/test_prokaryotic_lineage.py:48  AssertionError: []

It is new, and it is on main

$ gh run list --branch main --workflow validate-strict.yaml
08-29T02:03  22673ba  failure     <-- and 22673ba passed as a PR
08-28T08:26  2031b7e  success
08-28T07:35  4a65c99  success
08-28T06:49  d11d1ad  success
08-27T19:16  80b882c  success

The five runs before it succeeded, so the download broke some time between
08-28T08:26 and 08-29T02:03. Any PR opened now inherits a red validate-strict
that has nothing to do with its contents — which is how a real failure gets
waved through as "the flaky one".

Why it can happen at all

validate-strict.yaml has no actions/cache step:

$ grep -c actions/cache .github/workflows/validate-strict.yaml
0
$ grep -c actions/cache .github/workflows/label-correspondence.yaml
1

label-correspondence caches ~/.data/oaklib across runs; validate-strict
re-downloads the NCBITaxon database every time and has nothing to fall back on.
So the lane that runs the whole test suite is the one with no protection against
the download it depends on.

The library code is already correct about this — domain_of's docstring says
None covers "no adapter, a lookup failure, and a taxon above every domain",
and outside_gtdb_scope is deliberately one-directional so a caller "may
downgrade on False but must never upgrade on it". tests/test_ncbi_domain_scope.py
even has test_an_unavailable_adapter_degrades_rather_than_guesses. It is the
other tests in those modules that assume the adapter is present.

Two fixes, and they are not alternatives

1. Cache OAK in validate-strict, as label-correspondence already does.
   Removes most occurrences; does not help a cold cache or an upstream outage.
2. Make the adapter-dependent assertions skip, with a reason, when the adapter
   is unavailable — asserting a lookup result without the lookup is not a test
   of anything. This must be done carefully: a blanket skip would hide a real
   regression, so the skip should be narrow and label-correspondence (which
   caches, and therefore usually has the adapter) stays the lane that genuinely
   exercises it.

Doing only (1) leaves the suite red whenever the download fails. Doing only (2)
leaves every run paying for a download it could have cached.

Acceptance test

With the network to the OAK download blocked, uv run pytest tests/ reports
skips with a stated reason rather than failures, and a run with the cache warm
exercises the assertions normally.

[realmarcin]

Two corrections to this issue after trying to fix it. Both make it larger than
written.

1. The download is broken upstream, not on the runner

Verified from a developer machine, not just CI:

$ ranged GET https://s3.amazonaws.com/bbop-sqlite/ncbitaxon.db.gz
HTTP Error 403: Forbidden

Both HEAD and a Range: bytes=0-1023 GET return 403. So the artifact OAK
fetches for sqlite:obo:ncbitaxon is unavailable to anyone without a warm cache
— it is not a runner network fault, and caching cannot fix it on its own: a
cache only warms from a download that succeeded, and this one does not.

My local runs pass only because ~/.data/oaklib was populated before the
outage. That is worth stating plainly: a green local suite right now is evidence
about my cache, not about the code.

2. It is not 12 tests in 2 modules; there are at least three adapter

entry points

PR #702 gated the assertions that fail when
communitymech.validators.ncbi_domain._adapter() returns None — measured by
forcing that function to None and reading off the failures. CI then failed on a
different set:

tests/test_shared_taxon_ids.py       9 failures
tests/test_prokaryotic_lineage.py    test_the_gate_fires_through_validate_strict

because the measurement was scoped to one copy of the adapter. There are
three, each independently constructing the same thing:

src/communitymech/validators/ncbi_domain.py:57      get_adapter("sqlite:obo:ncbitaxon")
src/communitymech/validators/shared_taxon_ids.py:187 get_adapter("sqlite:obo:ncbitaxon")
src/communitymech/cross_repo_environment.py:238      get_adapter(f"sqlite:{db}")

Same shape as #690 and #697: N copies of one fact, fixed in one place, and the
others carry on. The right fix is one shared accessor the whole repository
goes through, so "is NCBITaxon available?" has a single answer and a single
place to degrade.

3. A design tension worth deciding deliberately

Some of these tests fail loudly on purpose when the ontology is missing:

tests/test_shared_taxon_ids.py::test_the_committed_kb_is_clean
    AssertionError: NCBITaxon unavailable; this proves nothing

That is a considered choice — refusing to pass vacuously — and it is the same
instinct as [gtdb-domain] NCBITaxon is unavailable, so the prokaryote-only check was skipped, not passed. Converting those to skips would be overriding
someone's decision, so #702 did not touch them.

The question this issue should settle: is "the ontology is unavailable" a
skip or a failure for this repository? Both are defensible. What is not
defensible is the current state, where it is a skip in two modules, a loud
failure in a third, and an unexplained red in a fourth.

Suggested shape

1. One shared ncbitaxon_adapter() accessor; the three call sites go through it.
2. One decision about skip-vs-fail, applied everywhere through that accessor.
3. Keep the OAK cache added in fix: stop the cache-name loop, share the NCBITaxon adapter, write the protocol (#697, #700, #696, #704, #705) #702 — it removes the common case even though it
   cannot help a cold cache during an outage.
4. Pin or vendor the ontology if the availability of bbop-sqlite is going to
   decide whether this repository's CI is green.

[realmarcin]

Fixed in 851b39a (PR #702).

Two things, and the second matters more than the first.

The cache step. validate-strict and python-compatibility now cache ~/.data/oaklib, as label-correspondence always did. (#707 tracks the fact that the fixed key never rotates.)

The shared accessor, which is what actually fixed it. ncbi_domain and shared_taxon_ids each built their own adapter from sqlite:obo:ncbitaxon. The copies always agreed, so the duplication broke a measurement rather than behaviour: I gated the tests that fail when one returns None, and CI failed on thirteen callers of the other. communitymech.ontology_adapters is now the single construction site and ncbitaxon_available() the single question, re-measured against the whole suite with the ontology unreachable rather than against one module. tests/test_ncbitaxon_adapter_is_shared.py asserts the identity, the single selector literal, and that the gated modules skip rather than fail — the last hermetically, via an empty PYSTOW_HOME plus an unroutable proxy, so it needs no network and cannot pull 13 GB the day the 403 lifts.

The download is still broken upstream. s3.amazonaws.com/bbop-sqlite/ncbitaxon.db.gz returns 403 on both HEAD and a ranged GET, verified from a developer machine. Caching cannot repair that — a cache only warms from a download that succeeds. CI is green because the checks skip loudly, and #708 tracks that the #292 and #365 gates are consequently off until the artefact returns.