Ground GENE_OR_PROTEIN nodes via kg-microbe UniProt transform (+42 nodes)

[realmarcin]

Summary

Uses kg-microbe's merged-kg_uniprot_nodes.tsv (≈2.2 GB, 19.8 M UniprotKB: rows) as the source-of-truth label index for TraitMech's ungrounded GENE_OR_PROTEIN nodes. Streams the file once, matches each TraitMech residual GENE_OR_PROTEIN label against UniProt names, and grounds 42 nodes via 27 representative UniProt CURIEs.

Per user request: "use the kgm uniprot transform for grounding and reference" — the path used is /Users/marcin/Documents/VIMSS/ontology/KG-Hub/KG-Microbe/kg-microbe/merged-kg_uniprot_nodes.tsv.

New tooling

scripts/match_uniprot_to_proteins.py — one-off matcher. Streams the kg-microbe UniProt nodes file, collects per-label candidates (cap 500/label), and picks a single representative per label via a two-tier selector:

1. Tier 1 (best) — UniProt name (cleaned of trailing parens) equals the TraitMech label exactly. Among those, alphabetically-first CURIE for determinism.
2. Tier 2 — name ends with " <label>" as the final whitespace token (e.g. "Polarized growth protein Scy" for label scy). Prefer SHORTEST such name (fewer modifier words like chaperone, maturation protein, assembly factor), alphabetic CURIE as tiebreaker.
3. Otherwise — skip (too ambiguous to ground cleanly).

Also skips a hand-curated set of abstract-category labels that shouldn't be grounded to a specific UniProt entry: gene product, virulence factors, chaperone proteins, thermostable proteins, salinity-adaptation genes, cold-shock proteins, proton export pumps and antiporters, membrane transporters, gliding motility machinery, rod complex.

Mapping additions

mappings/node_grounding.tsv grows 47 → 74 rows. 27 UniProt mappings — examples:

TraitMech label	UniProt CURIE	UniProt name
mreb	UniprotKB:A0A1B1UYY2	MreB
ftsz	UniprotKB:C0LUM8	FtsZ
diviva	UniprotKB:Q1IYG2	DivIVA
atp synthase	UniprotKB:A0A415TT77	ATP synthase
na+/h+ antiporter	UniprotKB:A0A068T423	Na+/H+ antiporter
superoxide dismutase	UniprotKB:A0A009QPW9	Superoxide dismutase (Cu-Zn)
methyl-coenzyme m reductase	UniprotKB:A0A099T5Q9	Methyl-coenzyme M reductase
pqq-dependent methanol dehydrogenase	UniprotKB:A0A4U8YZA6	PQQ-dependent methanol dehydrogenase (Alpha subunit)
crescentin	UniprotKB:A0A2N9AY16	Crescentin (CreS)
pbp2b	UniprotKB:A0A892RPK7	Penicillin-binding protein PBP2B

Full per-label audit at reports/uniprot_match_candidates.tsv.

Corpus impact

	Before	After
Nodes grounded	622	664 (+42, 50% → 53%)
Nodes residual	630	588 (−42)
Distinct (label, type) residuals	503	476 (−27)
Mappings TSV (rows)	47	74 (+27)

Caveats

• The 27 representatives are species-specific UniProt entries, not canonical family-level groundings. PRO (Protein Ontology) would be the cleaner target for protein families but no local PRO snapshot is available; this PR picks the representative-UniProt approach explicitly authorized by the user.
• 41 of 93 GENE_OR_PROTEIN labels had at least one UniProt name match; 27 picked after the tier-1/2 filter; 14 had matches but failed the strict tier criteria (mostly chaperone / assembly factor / domain-containing protein candidates); 10 hand-skipped as abstract categories; 42 had no UniProt match at all (likely TraitMech-specific paraphrases or non-protein labels — salt-in strategy, weak/absent shape-determining cytoskeleton, etc.).
• The matcher path is currently hardcoded to a local kg-microbe checkout; future runs need either an env var or a copy of merged-kg_uniprot_nodes.tsv into data/raw/. Left as-is for this PR since the script is one-off.

Verified locally

$ uv run python scripts/match_uniprot_to_proteins.py --apply
  labels with at least one match: 41 / 93
  representatives picked: 27
  abstract-category labels skipped: 10
  appended 27 rows to mappings/node_grounding.tsv

$ just ground-nodes --apply
  files modifiable:    [N]
  nodes grounded:      42
  residual nodes:      588 across 476 distinct (label, type) keys

$ just ground-nodes        # second pass, idempotency check
  nodes grounded:      0

$ just validate-strict
  files scanned:      357
  files with ERROR:   0

Test plan

• Idempotency: re-run of ground-nodes produces 0 additional groundings
• validate-strict clean
• Audit trail in reports/uniprot_match_candidates.tsv
• CI re-runs validate-strict on YAML diff

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds a one-off matcher that uses kg-microbe’s UniProt node export to select representative UniProt CURIEs for previously-ungrounded GENE_OR_PROTEIN causal-graph nodes, then updates the curated mapping table and applies the resulting groundings across affected trait YAMLs (with an audit report and updated residual report).

Changes:

• Add scripts/match_uniprot_to_proteins.py to stream merged-kg_uniprot_nodes.tsv, generate candidate matches, and (optionally) append selected representatives to mappings/node_grounding.tsv.
• Expand mappings/node_grounding.tsv with 27 new UniProt representative mappings and commit the audit output reports/uniprot_match_candidates.tsv.
• Apply new grounding: CURIEs to GENE_OR_PROTEIN nodes across multiple trait YAML causal graphs; update reports/node_grounding_residual.tsv accordingly.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
scripts/match_uniprot_to_proteins.py	New streaming matcher to find UniProt candidates and append representative mappings.
reports/uniprot_match_candidates.tsv	New audit TSV listing candidate counts and chosen representatives per label.
reports/node_grounding_residual.tsv	Updated residual report after applying new mappings/groundings.
mappings/node_grounding.tsv	Adds 27 GENE_OR_PROTEIN → UniProt representative mappings.
data/traits/physiology/methylotrophic.yaml	Adds UniProt groundings for methanol dehydrogenase nodes; appends curation event.
data/traits/physiology/methanotrophic.yaml	Adds UniProt groundings for methane monooxygenase / methanol dehydrogenase; appends curation event.
data/traits/physiology/hydrogenotrophic.yaml	Adds UniProt grounding for hydrogenase; appends curation event.
data/traits/physiology/chemolithotrophic.yaml	Adds UniProt grounding for ammonia monooxygenase; appends curation event.
data/traits/physiology/carboxydotrophic.yaml	Adds UniProt groundings for CODH / molybdenum hydroxylase; appends curation event.
data/traits/morphology/spirochete_shaped.yaml	Adds UniProt grounding for FlaB; appends curation event.
data/traits/morphology/rod_shaped.yaml	Adds UniProt groundings for MreB and FtsZ; appends curation event.
data/traits/morphology/ovoid_shaped.yaml	Adds UniProt grounding for DivIVA; appends curation event.
data/traits/morphology/oval_shaped.yaml	Adds UniProt grounding for PBP2b; appends curation event.
data/traits/morphology/motility.yaml	Adds UniProt grounding for type IV pilus; appends curation event.
data/traits/morphology/motile.yaml	Adds UniProt grounding for type IV pilus; appends curation event.
data/traits/morphology/helical_shaped.yaml	Adds UniProt grounding for CcmA; appends curation event.
data/traits/morphology/filament_shaped.yaml	Adds UniProt groundings for DivIVA and Scy; appends curation event.
data/traits/morphology/ellipsoidal.yaml	Adds UniProt groundings for PBP2b and DivIVA; appends curation event.
data/traits/morphology/crescent_shaped.yaml	Adds UniProt grounding for crescentin; appends curation event.
data/traits/morphology/coccobacillus_shaped.yaml	Adds UniProt grounding for MreB; appends curation event.
data/traits/morphology/cell_shape.yaml	Adds UniProt groundings for MreB, FtsZ, crescentin; appends curation event.
data/traits/morphology/brown_pigmented.yaml	Adds UniProt grounding for 4-hydroxyphenylpyruvate dioxygenase; appends curation event.
data/traits/morphology/bacillus_shaped.yaml	Adds UniProt groundings for MreB and FtsZ; appends curation event.
data/traits/metabolism/substrate_level_phosphorylation.yaml	Adds UniProt grounding for acetate kinase; appends curation event.
data/traits/metabolism/respiration.yaml	Adds UniProt grounding for ATP synthase; appends curation event.
data/traits/metabolism/methanogenesis.yaml	Adds UniProt grounding for methyl-coenzyme M reductase; appends curation event.
data/traits/metabolism/electron_transfer.yaml	Adds UniProt groundings for redox protein and c-type cytochrome; appends curation event.
data/traits/metabolism/aerobic_respiration.yaml	Adds UniProt groundings for cytochrome c oxidase and ATP synthase; appends curation event.
data/traits/environment/obligately_alkaphilic.yaml	Adds UniProt grounding for Na+/H+ antiporter; appends curation event.
data/traits/environment/neutrophilic.yaml	Adds UniProt grounding for cation/proton antiporter; appends curation event.
data/traits/environment/hyperthermophilic.yaml	Adds UniProt grounding for reverse gyrase; appends curation event.
data/traits/environment/facultatively_alkaphilic.yaml	Adds UniProt grounding for Na+/H+ antiporter; appends curation event.
data/traits/environment/alkaphilic.yaml	Adds UniProt grounding for Na+/H+ antiporter; appends curation event.
data/traits/environment/alkalotolerant.yaml	Adds UniProt grounding for cation/proton antiporter; appends curation event.
data/traits/environment/aerotolerant.yaml	Adds UniProt grounding for superoxide dismutase; appends curation event.

Comments suppressed due to low confidence (5)

scripts/match_uniprot_to_proteins.py:52

• KG_UNIPROT_NODES is hardcoded to a local absolute path ("/Users/marcin/..."), which makes this script non-runnable for other contributors and in CI. Please make the UniProt-nodes TSV path configurable (e.g., --kg-uniprot-nodes arg and/or env var), and keep the repo default either unset or relative to the repo (with a clear error if missing).
  scripts/match_uniprot_to_proteins.py:98
• build_regex() will compile a pattern that matches the empty string if labels is empty (because (?:|...) becomes (?:)), which would make regex.finditer(name) yield zero-length matches at every position and effectively hang the scan. Please guard against an empty labels list (e.g., return early in main() with a clear message, or raise in build_regex()).
  scripts/match_uniprot_to_proteins.py:206
• The match_count reported to reports/uniprot_match_candidates.tsv is len(cands), but cands is capped at MAX_MATCHES_PER_LABEL, so counts of 500 are ambiguous (could mean 500 total, or “>=500”). Please track total match occurrences separately from the stored candidate list (e.g., total_matches[label]), and write both to the report so ambiguity/skip thresholds can be reasoned about correctly.
  scripts/match_uniprot_to_proteins.py:225
• --apply appends to mappings/node_grounding.tsv unconditionally, so re-running the script will duplicate rows (and make diffs/noise harder to manage). Please load existing mappings first and skip (or update in-place) any (label, node_type) keys that already exist, and consider sorting/deduplicating output for deterministic reruns.
  mappings/node_grounding.tsv:52
• The per-row notes string claims the representative was selected via “name-ends-with-label + alphabetic-first CURIE”, but the actual selection logic described/implemented in the matcher includes an exact-match tier and a “shortest suffix-hit name” preference with CURIE only as a tiebreaker. Please update the notes text to accurately reflect the selection criteria (or reference the script/report) so the mapping provenance is not misleading.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.