Resolve the CURIEs the research reports suggest (#243)

[realmarcin]

Closes #243.

The sweep's candidate-node tables pair a human label with an ontology identifier, and nothing resolved them. A sample against OLS during the #241 review found the long tail materially wrong. This checks all of them against OAK and writes a triage backlog to reports/research_grounding_drift.tsv.

What the corpus actually contains

1733 (id, label) pairs checked against an ontology, across 354 reports:

verdict	n	meaning
OK_LABEL	952	label is the canonical or a synonym
OK_IN_ROW	289	the row says what the id means, just elsewhere in the row
DRIFT	425	the row never states what the id actually means
OBSOLETE	39	resolves to a deprecated term
UNRESOLVED	28	the id is not in the ontology at all
UNKNOWN_PREFIX	10	a prefix outside the grounding policy (SO:, EFO:)

467 distinct actionable suggestions, written to reports/research_grounding_backlog.tsv — ranked by verdict then least-similar-first, deduplicated, with an occurrence count so one decision closes every site. reports/research_grounding_drift.tsv keeps every occurrence for locating them afterwards.

CHEBI:14341 and CHEBI:42899 are both offered as "ectoine" and neither exists. GO:0009405 is offered as "pathogenesis" — the label agrees with GO's, and the term is obsolete, which is precisely the case a label comparison alone cannot see.

doi, ec, metpo, ncbitaxon and traitmech prefixes are named as having no OAK adapter rather than skipped by omission, so a typo like CHBEI: still surfaces as unknown instead of vanishing. Matching is casefolded, because the reports spell prefixes inconsistently.

A report, not a gate

Exits 0 whatever it finds, and is deliberately not in just qc:

• The reports are provider output. Nobody is going to hand-edit 353 of them, so failing a build on their contents would gate work on data no one intends to correct in place.
• The extraction from prose tables is heuristic, so some findings are judgement calls.

The blocking gate stays where the curated data is — just validate-products over mappings/*.tsv, which is where these suggestions land if a curator accepts one. The value here is seeing "this does not resolve to what the report claims" before that lift.

Why not extend the existing validator

scripts/validate_id_label_correspondence.py does exactly this check for the curated TSVs and was the obvious host. It is vendored byte-identical across the Mech repos and diffed against CultureMech by the vendored-sync job, so editing it here fails CI; the propagation path runs through the hub. This is a TraitMech-local script that mirrors its config vocabulary — same adapters, same canonical-or-synonym policy — so the two read alike.

Six defects found by running it and by review, not by reading it

Per the canary rule, one report first (13 pairs, rows verified on disk, and it found the obsolete GO:0009405), then the full corpus. The full run surfaced both problems:

• Fatty-acid shorthand parsed as CURIEs. C16:0, C18:1 produced 78 bogus UNKNOWN_PREFIX findings that buried the real ones. The prefix pattern is now letters-only; no ontology prefix these reports use carries a digit. 78 → 35.
• DRIFT mixed two populations. Genuine mis-groundings sat beside lexical variants of the right term, because CHEBI does not list "proton" as a synonym of hydron and fumarate is not literally fumaric acid. Each row now carries a similarity score and the output sorts least-similar first, so ectoine/(-)-beta-caryophyllene leads and fumarate/fumaric acid trails. There is no cutoff — the score orders, it does not filter.

Verification

293 tests pass  (20 new)
just qc         exit 0

Review also found four more, all fixed here: the ranked backlog existed only in stdout (#263); an adapter failure was reported as "id not in the ontology", so a failed semsql download would have rewritten the committed TSV to ~1200 findings at exit 0 (#262, #265); case-sensitive prefix matching sent 25 truncated doi:10 citation fragments into the backlog (#261); and OBSOLETE scored 1.0 under an ascending sort, burying all 39 obsolete findings below every DRIFT — including the GO:0009405 this body leads with (#264).

Left open and filed: #266, merged terms reading as "never existed" when the successor id is the answer.

The tests stub the adapter and cover extraction and verdicts — where both defects were. test_a_lexical_variant_scores_above_a_wholesale_mismatch pins the ordering the report depends on.

🤖 Generated with Claude Code

[culturebot-reviewer]

Claude finished @realmarcin's task in 4m 35s —— View job

---

Review

The premise is right and the findings I spot-checked are real — CHEBI:62202 is not pyocyanin, GO:0009405 really is obsolete, and the two ectoine CHEBI ids really don't exist. The similarity score is the right answer to the fumarate/fumaric acid problem. Three things need fixing before this is a backlog someone can work from.

---

🟡 doi: and metpo: slip past NO_ADAPTER — the prefix check is case-sensitive

NO_ADAPTER holds "DOI" and "METPO"; the regex captures the literal case found in the text. scripts/audit_research_groundings.py:224-228 compares them directly, so doi:10.1371/journal.pbio.3001443 truncates to doi:10 and lands in UNKNOWN_PREFIX.

25 of the 35 UNKNOWN_PREFIX rows are doi:10 — every one from an evidence table, so claimed_label is a graph subject like mreb, cgld, calcium dependence. reports/research_grounding_drift.tsv:1509 shows the metpo: half: lowercase metpo:1000650 was taken as the claimed label for CHEBI:16183 and only escaped a DRIFT verdict because "methane" happened to appear later in the row.

This is the C16:0 failure mode again, and the PR body's claim that "DOI … METPO … are named as having no OAK adapter rather than skipped by omission" isn't true as written. ~25 of the headline 490 distinct actionable suggestions are DOI fragments. Casefold the prefix before both membership tests. Fix this →

🟡 An adapter failure is reported as "not in the ontology"

Ontologies.lookup (:136-147) catches every exception and returns None, which classify turns into UNRESOLVED. The comment calls that "the honest verdict" — it isn't. A failed semsql download, or a 0-byte sqlite stub, would rewrite the committed TSV to ~1200 UNRESOLVED rows, exit 0, and look like a catastrophic corpus finding.

scripts/validate_id_label_correspondence.py — the script this one says it mirrors — draws exactly this distinction, with ADAPTER_ERROR and SKIPPED_EMPTY_ADAPTER as verdicts separate from ID_NOT_FOUND, and its docstring says why. Since the TSV is committed, the failure is durable and silent. Let an adapter-open failure be its own verdict (or abort); reserve UNRESOLVED for an adapter that opened and returned no label.

🟡 The ranked, deduplicated backlog is only in stdout

distinct (:254-258) is sorted least-similar-first and deduplicated, then used solely to print 15 lines. The TSV gets rows — corpus order, no dedup, with the 1241 OK_* rows interleaved. Compare reports/research_grounding_drift.tsv:16 (0.67) to :22 (0.20).

So "the output sorts least-similar first, so ectoine leads and fumarate trails" describes the console summary, not the artifact. The 490-item backlog the PR is named for exists nowhere on disk. Either sort/dedup the TSV, or emit distinct as a second file.

---

🔵 Smaller things

• OBSOLETE sorts to the bottom. classify returns 1.0 for obsolete (:185), and the ordering is ascending, so all 39 obsolete findings rank below every DRIFT. The printed top-15 is entirely 0.00 entries — GO:0009405, the case the PR body leads with as the one a label check can't see, never appears. It's a distinct verdict, not a similarity; consider sorting on (verdict_rank, score).
• Obsolescence is inferred from the label string. label.lower().startswith("obsolete") is a GO/OBO convention, not a guarantee — CHEBI deprecates without relabelling, so an obsolete CHEBI id with a matching label scores OK_LABEL. OAK exposes deprecation directly. Relatedly, merged terms come back as UNRESOLVED (GO:0015992, GO:0009878), which reads as "never existed" when the successor is the answer a curator wants.
• The TSV is a tracked derived report outside audit-derived-reports. That recipe's own comment documents what goes wrong (cellobiose sat stale "for weeks… exactly the size of error nobody notices"). Network-dependent generation may make it genuinely unwireable — if so, say that in the justfile comment, and consider stamping the run date and report count into the file so its vintage is visible.
• Recipe naming. Every other audit-* recipe is a qc member; the non-gating baseline is report-label-drift. audit-research-groundings reads as a gate it deliberately isn't.
• scripts/audit_research_groundings.py:249-253 says "deduplicated by (curie, claimed_label)" twice, and the key is actually a 5-tuple. :271 labels sum(counts.values()) "checkable pairs" while including UNKNOWN_PREFIX, which by definition wasn't checked.

No concerns on the data side: nothing here touches data/traits/** or the curated mapping tables, so no history record is expected and validate-products is unaffected.

---

• Read repo conventions
• Read script, tests, justfile
• Spot-check the report output
• Post review
  · branch feat/research-grounding-drift

[culturebot-reviewer]

Premise is right and the findings I spot-checked are real. Three blockers:

1. The NO_ADAPTER prefix check is case-sensitive, so lowercase doi: and metpo: fall through — 25 of the 35 UNKNOWN_PREFIX rows are doi:10 fragments from DOIs in evidence tables. Same failure mode as C16:0, and it inflates the 490-item headline.
2. Ontologies.lookup turns any adapter exception into UNRESOLVED. A failed semsql download would rewrite the committed TSV to ~1200 bogus unresolved rows and exit 0. The vendored validate_id_label_correspondence.py keeps ADAPTER_ERROR separate from ID_NOT_FOUND for exactly this reason.
3. The sorted, deduplicated backlog exists only as 15 lines of stdout. The TSV is written in corpus order with the OK rows interleaved, so the artifact the PR is named for isn't on disk.

Plus smaller items (OBSOLETE sorts last because it scores 1.0; obsolescence inferred from the label string; the tracked TSV has no staleness check; recipe naming) — full detail in the review comment.

[culturebot-reviewer]

Claude finished @realmarcin's task in 3m 29s —— View job

---

Review

Two of the three blockers are fully fixed and I verified them against the regenerated artifacts. The third is fixed for the case it was reported under but not for the case the new comment claims it covers.

---

✅ Confirmed fixed

• Case-sensitive prefix check. _ADAPTERS_CF / _NO_ADAPTER_CF (scripts/audit_research_groundings.py:75-76) casefold both lookups, and the artifacts were regenerated: UNKNOWN_PREFIX 35 → 10, and all 10 remaining are real SO: / EFO: suggestions genuinely outside the policy (reports/research_grounding_drift.tsv:624, :1149), not doi:10 fragments. Backlog contains zero DOI rows.
• The backlog is on disk. reports/research_grounding_backlog.tsv is ranked by (VERDICT_RANK, similarity), deduplicated, and carries an occurrence count. test_the_backlog_artifact_exists_and_is_ranked pins the ordering against the committed file, which is the right shape for a test of a tracked derived artifact.
• Smaller items. OBSOLETE now outranks DRIFT by verdict rather than being buried at 1.0; _deprecated reads owl:deprecated off entity_metadata_map — I checked that this is the predicate key semsql actually stores (oaklib/datamodels/vocabulary.py:105, DEPRECATED_PREDICATE), so it does what it claims; the vintage line stamps report/pair/finding counts; the recipe is report-research-groundings with the naming rationale in the justfile comment; checked now excludes UNKNOWN_PREFIX.

---

🟡 The empty-adapter case still reports ~1200 ids as "not in the ontology"

Ontologies.lookup's docstring (scripts/audit_research_groundings.py:147-149) says ADAPTER_ERROR means "the lookup could not be performed at all — a failed semsql download, an empty sqlite stub, an OAK change."

It doesn't cover the stub. ADAPTER_ERROR is only returned when get_adapter or adapter.label raises (:159-168). A 0-byte or schema-only sqlite opens without error and returns label() == None for every id — so it lands on result = None → UNRESOLVED, which is precisely the failure the blocker described: the committed TSVs silently rewritten to ~1200 rows that read as a catastrophic corpus finding, exit 0.

validate_id_label_correspondence.py — the script this one mirrors — separates LOAD_FAILED from EMPTY_ADAPTER for exactly this reason, and its _is_empty (:264-286) is an O(1) probe: next(iter(adapter.entities()), None) is None, with a positive stub confirmation when the probe itself raises, so a partially-migrated live ontology isn't masked. Mirroring that at adapter-open time is a few lines. Either do that, or narrow the docstring to the exception path it actually handles — the current wording is the more dangerous half, because it tells the next reader the case is covered.

Fix this →

---

🔵 Smaller things

• The PR body no longer matches the artifacts. It states 1768 checkable pairs, 490 distinct suggestions and UNKNOWN_PREFIX 35; the regenerated files say 1733 / 467 / 10, and the verdict table in the body is the old one. The DOI/METPO sentence is now true but describes a different number. Since the body is the record for Suggested ontology groundings in the research reports are unvalidated, and materially wrong in the long tail #243, it wants the current table.
• Merged terms still read as "never existed." GO:0009878 and GO:0015992 sit in the backlog as UNRESOLVED, which tells a curator the id is bogus when the answer they want is the replacement. OAK exposes term replaced by / obsoletes_migration_relationships. Genuinely optional — but it's the difference between a backlog row someone can close and one they have to re-research.

Nothing here touches data/traits/** or the curated mapping tables, so no history/ record is expected and validate-products is unaffected.

I could not run the test suite — pytest was not on the sandbox allowlist for this job, so the 14 new tests are reviewed by reading only. Everything I assert about the artifacts above comes from reading the committed TSVs directly.

---

• Read repo conventions
• Read the updated script, tests, justfile
• Verify blocker 1 (prefix casefolding) against regenerated output
• Verify blocker 2 (adapter error vs unresolved)
• Verify blocker 3 (backlog artifact)
• Check the smaller items
• Post verdict
  · branch feat/research-grounding-drift

[culturebot-reviewer]

Two of the three blockers are fully fixed and verified against the regenerated artifacts: the prefix casefolding (UNKNOWN_PREFIX 35 -> 10, remaining 10 are real SO:/EFO: suggestions, zero DOI rows in the backlog), and the ranked/deduplicated backlog now existing as reports/research_grounding_backlog.tsv with an ordering test. The smaller items — OBSOLETE outranking DRIFT by verdict, owl:deprecated read off entity_metadata_map, the vintage stamp, the report-* naming — are all done.

One 🟡 remains. Ontologies.lookup's docstring (scripts/audit_research_groundings.py:147-149) says ADAPTER_ERROR covers 'an empty sqlite stub', but it only fires when get_adapter or adapter.label raises. A 0-byte or schema-only semsql opens without error and returns label()==None for every id, so every pair falls through to UNRESOLVED — the exact scenario the blocker described: ~1200 bogus 'not in the ontology' rows written to a committed TSV, exit 0. validate_id_label_correspondence.py separates LOAD_FAILED from EMPTY_ADAPTER for this reason; its _is_empty (:264-286) is an O(1) entities() peek with a positive stub confirmation when the probe raises. Mirror that, or narrow the docstring to the exception path it actually handles — the current wording is the more dangerous half.

Two 🔵: the PR body still quotes the pre-fix numbers (1768/490/35 vs the artifacts' 1733/467/10), and merged terms (GO:0009878, GO:0015992) still read as UNRESOLVED where the replacement is what a curator wants.

Full detail in the comment. Nothing touches data/traits/** or the curated mapping tables, so no history/ record is expected.

[culturebot-reviewer]

Claude finished @realmarcin's task in 2m 59s —— View job

---

Review

The last blocker is fixed, and fixed properly rather than by narrowing the docstring. Approving.

---

✅ The empty-adapter case is closed

Ontologies._is_empty (scripts/audit_research_groundings.py:176-202) probes next(iter(adapter.entities()), None) at adapter-open time, caches per prefix, and returns ADAPTER_ERROR for every lookup under an empty prefix. Two things I specifically checked:

• The probe-raises path is not treated as empty (:193-196) — empty = False, with the reason on stderr. That is the right call and the one that's easy to get backwards: a partially-migrated live ontology fails entities() the same way a stub does, and calling it empty would suppress real findings instead of surfacing a broken tool. test_a_probe_that_raises_is_not_treated_as_empty pins it, and the stub returns a real label so the assertion is on the outcome, not the flag.
• The failure is now loud, not silent. :198-200 names the prefix on stderr, and main (:410-413) prints the corpus-wide warning. A 0-byte semsql now produces ADAPTER_ERROR rows plus two stderr lines rather than ~1200 rows reading as a catastrophic corpus finding.

test_an_empty_adapter_is_an_error_not_1200_missing_ids covers the positive case. I traced both tests against the code by hand — pool._adapters["go"] matches the casefolded key lookup passes to _adapter, so the stub is actually reached.

✅ Artifacts and body now agree

Counted directly out of the committed files rather than trusting the summary line:

• research_grounding_drift.tsv: 952 / 425 / 289 / 39 / 28 / 10 → 1733 checked, matching the vintage stamp and the PR body's table.
• research_grounding_backlog.tsv: 395 DRIFT + 36 OBSOLETE + 26 UNRESOLVED + 10 UNKNOWN_PREFIX = 467 distinct, matching the body's 467.
• GO:0009405 "pathogenesis" is now at backlog line 45 with 3 occurrences — visible, which was the point of the verdict-first ranking.

The last commit touches only the script and tests, and _is_empty cannot change a verdict for a non-empty adapter, so the artifacts being unchanged is what should have happened.

---

🔵 One thing, still optional

Merged terms remain UNRESOLVED. GO:0015992 accounts for 7 of the 26 distinct unresolved rows (proton motive force, proton transport, hydrogen oxidation, …) and GO:0009878 for 2 — a third of that bucket is two merged ids, and the row tells a curator "this id is bogus" when the answer they want is the replacement. OAK exposes term replaced by. Genuinely optional, and reasonable to leave for a follow-up now that the backlog is on disk and rankable.

Not raising the leading # vintage line in the backlog TSV as an issue — no consumer in this repo globs reports/*.tsv, and the test already skips comment lines.

---

I could not run the test suite — pytest is not on this job's sandbox allowlist, so the 16 tests are reviewed by reading and by tracing the stubs against the code. Every count above comes from reading the committed TSVs.

Nothing here touches data/traits/** or the curated mapping tables, so no history/ record is expected and validate-products is unaffected.

• Read the new commit (empty-adapter fix)
• Verify the remaining 🟡 is closed
• Re-check the artifacts against the body
• Post verdict
  · branch feat/research-grounding-drift

[culturebot-reviewer]

The last blocker is closed properly rather than by narrowing the docstring. _is_empty (scripts/audit_research_groundings.py:176-202) probes one entity at adapter-open time, caches per prefix, and returns ADAPTER_ERROR for every lookup under an empty prefix — and deliberately does NOT treat a raising probe as empty, which is the half that's easy to get backwards (a partially-migrated live ontology fails the same way, and calling it empty would suppress real findings). Both directions are tested, and the failure is now loud on stderr plus a corpus-wide warning.

Verified the artifacts by counting them rather than trusting the summary: drift TSV is 952/425/289/39/28/10 = 1733 checked, backlog is 395+36+26+10 = 467 distinct, both matching the vintage stamp and the updated PR body. GO:0009405 now sits at backlog line 45 with 3 occurrences, which is what the verdict-first ranking was for. The last commit touches only script and tests, and _is_empty cannot change a verdict for a non-empty adapter, so unchanged artifacts is the correct outcome.

One remaining 🔵, optional: merged terms still read as UNRESOLVED. GO:0015992 is 7 of the 26 distinct unresolved rows and GO:0009878 another 2 — a third of that bucket is two merged ids, where the replacement is the answer a curator wants. Reasonable as a follow-up.

I could not run pytest (not on this job's allowlist), so the tests are reviewed by reading and by tracing the stubs against the code. Nothing touches data/traits/** or the curated mapping tables, so no history/ record is expected and validate-products is unaffected.