Skip to content

Suggested ontology groundings in the research reports are unvalidated, and materially wrong in the long tail #243

Description

@realmarcin

Found reviewing #241. This is the finding that should gate the #183 backfill.

The 353 sweep reports each carry candidate-node tables pairing a human label with a CURIE — | infection thread | \GO:0009860` where applicable | ... |`. Nothing resolves those CURIEs or compares the label to the ontology. These tables are exactly what a curator copies from when adding grounded nodes.

Measured error rate

Extracted 602 distinct (CURIE, label) pairs from report table column 1 — 293 GO, 218 CHEBI, 46 METPO, 22 NCBITaxon, 22 ENVO, 1 PATO.

High-frequency pairs are fine. Checked the 80 most-repeated against OLS: 70 matched outright, and all 10 "mismatches" were legitimate exact synonyms — CHEBI:15378 "proton"/hydron, CHEBI:15379 "molecular oxygen"/dioxygen, CHEBI:18012 "fumarate"/fumaric acid. No errors.

The long tail is not. 428 of the 602 pairs occur exactly once. Sampled 60 of them against OLS with synonym-aware matching: 24 suspect, of which these are flatly wrong rather than a labelling quibble:

CURIE report label actual ontology label
CHEBI:10357 ectoine (-)-beta-caryophyllene
CHEBI:16134 sulfide ammonia
CHEBI:67078 autoinducer-2 (AI-2) delta(9)-tetrahydrocannabinolic acid
ENVO:01000992 cold stress / low temperature shower fixture
GO:0042599 magnetosome lamellar body
GO:0031471 chlorosome ethanolamine degradation polyhedral organelle
GO:0003918 relaxase (TraI) DNA topoisomerase type II activity
GO:0005727 plasmid extrachromosomal circular DNA
GO:0009860 infection thread pollen tube growth
GO:0055086 calcium homeostasis nucleobase-containing small molecule metabolic process

Three more point at obsolete terms (GO:0000276, GO:0006542, GO:0015002). The remainder of the 24 are defensible near-misses (GO:0019253 "Calvin–Benson–Bassham cycle" vs reductive pentose-phosphate cycle is the same thing).

Extrapolating the hard-error share of that sample across 428 singleton pairs puts roughly 80-100 wrong CURIE suggestions in the corpus. A separate signal agrees: 70 CURIEs appear with more than one distinct label across the corpus — GO:0042710 is labelled both "biofilm formation" and "horizontal gene transfer".

Why the reports themselves are not the bug

These are research inputs, faithfully captured from the provider. The defect is that they carry hallucinated identifiers with no validation layer between them and the curated tables. The reports should not be hand-edited.

Fix — extend the machinery that already exists

scripts/validate_id_label_correspondence.py with conf/id_label_targets.yaml already does precisely this check (OAK adapters for CHEBI/GO/ENVO/PATO/RO, policy: canonical_or_synonym, curator-accepted exceptions:) — but only over kind: tabular TSV targets, mappings/node_grounding.tsv and mappings/predicate_grounding.tsv.

Add a kind: markdown_table target over research/traits/**/*-deep-research-*.md, and surface it the way just report-label-drift does — non-blocking, as a triage report. Making it blocking would be wrong: the gate would fail on provider output nobody intends to correct in place. The point is that a curator sees "this suggestion does not resolve to what the report claims" before lifting it into mappings/node_grounding.tsv, where the existing blocking gate takes over.

Blocks the evidence-backed edge backfill in #183.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions