From 6df34fb7785358af9f6b72f215a579e52217258f Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Mon, 3 Aug 2026 10:11:29 -0700 Subject: [PATCH 1/3] Propose absence semantics: one diagnosis, three different remedies (#294, #304, #307) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The three issues share a diagnosis — the schema can say "has value X" or "is empty", so empty is overloaded across at least four meanings: not done, impossible, undecidable from the source, and deliberately excluded. They do NOT share a remedy, which is the proposal's main structural claim. #294 is missing vocabulary, #307 is a missing slot, and #304 is not a schema problem at all — it is auditor logic reading a fine schema wrongly. Bundling them would drag a code fix into a data migration for no benefit, so each is proposed separately and they can be taken in any order. Measured rather than asserted, on 307 records / 1007 taxa: - #294: of 372 ungrounded taxa, 285 have no GTDB equivalent and 85 are ambiguous. Checking the remaining 5 is what makes the case — every one is an entry deliberately WITHHELD under #292, so the honest tally is 370 permanent, 2 withheld, and ZERO pending work. GTDB grounding is complete to its ceiling and the schema cannot say so. That is precisely why #276 read as ~40% outstanding when the ceiling is ~63%. - #304: 107 of 302 records with interactions (35%) are entirely COMMUNITY_LEVEL, which the auditor credits with no connections; and 931 of 1007 taxa (92%) are exempt from DISCONNECTED via abundance_level/functional_role. The exemption, not the connectivity check, is doing the work — so the rule does not mean what its name says, and it rewards unsourced metadata. - #307: 193 records carry engineering_design but only 40 carry free-text notes, so most have no place at all for a load-bearing negative result. The withheld finding changed the proposal while writing it: WITHHELD was added as a fifth enum value, and the "quick win" of grounding 5 stragglers was removed, because they are blocked on #292 rather than pending. The existing pin in tests/test_gtdb_withheld_groundings.py is a test compensating for vocabulary the schema lacks, which is itself an argument for the enum. Recommended order: the #304 COMMUNITY_LEVEL credit first (smallest, no migration, unblocks #273, removes the perverse incentive), then #294, then #307. §6 records what is deliberately NOT proposed, including a general absence framework — three instances is not enough evidence to design one. Decision requested per remedy; they are independent. Co-Authored-By: Claude Fable 5 --- proposals/absence_semantics.md | 215 +++++++++++++++++++++++++++++++++ 1 file changed, 215 insertions(+) create mode 100644 proposals/absence_semantics.md diff --git a/proposals/absence_semantics.md b/proposals/absence_semantics.md new file mode 100644 index 000000000..fb228e99d --- /dev/null +++ b/proposals/absence_semantics.md @@ -0,0 +1,215 @@ +# Absence semantics in CommunityMech (#294, #304, #307) + +Three open issues turn out to be one diagnosis. This proposal states it, quantifies each +instance, and recommends a remedy per instance — which is **not** the same remedy three +times. + +**Decision requested:** approve or reject each of the three remedies in §4. They are +independent; approving one does not commit to the others. + +--- + +## 1. The shared diagnosis + +The schema can say *"this slot has value X"* and *"this slot is empty"*, and nothing else. +Empty is therefore overloaded. In the records it currently means at least four different +things: + +| what the curator meant | example | +|---|---| +| **not done yet** | a taxon nobody has tried to ground | +| **impossible** | a virus, which GTDB will never classify | +| **undecidable from the source** | GTDB splits the NCBI taxon and the paper does not say which | +| **deliberately excluded** | a strain screened out of a consortium for antagonising its mutualist | + +A consumer reading a record cannot tell these apart, and neither can a gate. Worse, in one +case (§3.2) the audit actively *rewards* filling a slot with an unsourced guess, because a +filled slot is treated as more complete than an honest blank. + +The three issues are the three places this has already caused a concrete problem. + +--- + +## 2. Why one mechanism will not fix all three + +It is tempting to propose a single general "absence annotation" reused everywhere. That +would be wrong here, because the three instances differ in kind: + +- **#294** is a *missing vocabulary* problem. The information mostly exists — + `gtdb_ground.py` already computes three of the states — it simply is not persisted. +- **#307** is a *missing slot* problem. The information exists in the source and in the + curator's head, and there is nowhere to put it. +- **#304** is **not a schema problem at all**. It is auditor logic. The schema is fine; the + rule that reads it is wrong. + +Bundling them into one schema change would drag a code fix into a data migration for no +benefit. They are proposed separately below and can be done in any order. + +--- + +## 3. The three instances, with numbers + +### 3.1 #294 — GTDB grounding: 372 blanks, none of which are pending work + +Measured 2026-08-03 over 307 records / 1007 taxonomy entries: + +| | count | share of ungrounded | +|---|---:|---:| +| grounded | 635 of 1007 (63%) | — | +| **no GTDB equivalent** — eukaryote, virus, environmental pseudo-taxon, absent from mapping | 285 | 76% | +| **ambiguous** — GTDB splits the NCBI taxon with no majority | 85 | 23% | +| **groundable by the tool** | 5 occurrences | 1% | + +Checking what those last five are is what makes the case. Every one is an entry +**deliberately withheld** under #292 — *Bacteroides ovatus* on `NCBITaxon:821` +(*Phocaeicola vulgatus*) and `Nitrospiraceae bacterium` on `NCBITaxon:1236` +(Gammaproteobacteria), whose ids name a different organism, pinned by +`tests/test_gtdb_withheld_groundings.py` so a tool re-run cannot reinstate them. + +So the honest tally is **370 permanently ungroundable, 2 deliberately withheld, and zero +pending work** — three distinct states rendered as one blank. GTDB grounding is complete to +its ceiling and the schema cannot say so. This is exactly why #276 read as ~40% outstanding +work when the achievable ceiling is ~63%: the issue — which I wrote — mistook impossibility +for backlog, and nothing in the data could have corrected that reading. + +It also means #294 has a **fourth** state to represent, distinct from the other three: +grounding withheld pending an upstream correction. + +### 3.2 #304 — DISCONNECTED fires on the wrong criterion + +Two rules interact in `network/auditor.py`: + +1. `connected_taxa` is built only from `source_taxon`/`target_taxon` (lines ~189, ~229). A + `COMMUNITY_LEVEL` interaction has neither by design, so it contributes **no** + connections. **107 of 302 records with interactions (35%) are entirely + `COMMUNITY_LEVEL`** — structurally, a third of the KB has zero connected taxa. +2. A taxon carrying `abundance_level` **or** `functional_role` is exempt (line ~258). + **931 of 1007 taxa (92%) are exempt on this basis.** + +The exemption is doing essentially all the work. `DISCONNECTED` does not mean "this taxon +has no curated interaction"; it means "this taxon has no *pairwise* interaction **and** no +membership metadata" — a compound of connectivity and slot-completeness that the name does +not convey. + +**The consequence is a perverse incentive.** In PR #298 I had invented `abundance_level` +values (`AbundanceEnum` is quantitative — `DOMINANT` is ">1% relative abundance" — and the +paper reported no abundances). Review removed them, which is unambiguously correct, and the +immediate result was five new `DISCONNECTED` findings. A curator optimising against the +audit would put the guesses back. + +### 3.3 #307 — counter-selection has nowhere to live + +`SynCom_ARC` (CommunityMech:000314) is *defined* by an exclusion: candidate *Bacillus* +isolates that inhibited *Bradyrhizobium* were screened out, so the shipped community is the +subset of effective antifungal strains that spares the nitrogen-fixing mutualist. + +There were two places to put that, and both are bad: + +- **As an `ecological_interaction`** — machine-readable but false. Excluded and retained + isolates are indistinguishable at the genus-level grounding the source supports, so the + typed edge asserts that an ARC *member* antagonises the mutualist ARC exists to spare. +- **As `engineering_design.notes` prose** — accurate but invisible to any query. + +PR #305 chose prose. That was right for the record and means the fact is no longer +queryable. Screening is how most SynComs are built, so this recurs: 193 records carry +`engineering_design`, and only 40 carry free-text `notes` — the rest have no natural place +for a negative result at all. + +--- + +## 4. Proposed remedies + +### 4.1 #294 — add a grounding-status enum *(schema, small)* + +Add to `TaxonDescriptor`, alongside `gtdb_classification`: + +```yaml +gtdb_grounding_status: + range: GtdbGroundingStatusEnum + # GROUNDED | NO_GTDB_EQUIVALENT | AMBIGUOUS | WITHHELD | NOT_ATTEMPTED + required: false +``` + +`WITHHELD` is needed because §3.1 found two real instances of it: grounding is possible and +deliberately not applied, pending an upstream fix (#292). Without it those two collapse into +`NOT_ATTEMPTED`, which is what the pin in `tests/test_gtdb_withheld_groundings.py` currently +exists to prevent — a test compensating for vocabulary the schema lacks. + +`AMBIGUOUS` should carry the candidate GTDB taxa, since that is precisely what a curator +needs in order to resolve it. Either a companion multivalued `gtdb_ambiguous_candidates` +slot, or fold the status into the existing `GtdbClassification` class and allow it without +a `gtdb_id`. + +`gtdb_ground.py` already distinguishes GROUNDED, AMBIGUOUS and NO_GTDB_EQUIVALENT +internally, so populating those three is a mechanical pass rather than a curation effort; +only WITHHELD needs to be asserted by hand, and there are two of them. Add a gate afterwards asserting the status matches +what the tool computes, so it cannot drift — the shape used by `tests/test_enum_groundings.py`. + +**Cost:** one schema change, one datamodel regeneration, one backfill run, one test. +**Benefit:** ~63% coverage stops looking like ~63% completion. + +### 4.2 #304 — fix the auditor, change no data *(code, small)* + +Two independent changes: + +1. **Credit `COMMUNITY_LEVEL` interactions.** Treat every taxon in a record as connected by + a community-level interaction, which is what such an interaction asserts. This alone + removes the structural penalty on 107 records. +2. **Drop the `abundance_level`/`functional_role` exemption.** With (1) in place, the + exemption's original purpose — suppressing false positives on records that describe + membership without pairwise edges — is served properly rather than by proxy. + +Doing (1) without (2) is safe and strictly an improvement. Doing (2) without (1) would +surface a large backlog at once and should not be done alone. + +**This is prerequisite for #273.** That issue cannot decide whether to restore the network +gate while `DISCONNECTED` means something other than its name. + +### 4.3 #307 — add a counter-selection block *(schema, medium)* + +Add to `CommunityEngineeringDesign`: + +```yaml +counter_selection: + multivalued: true + range: CounterSelection # excluded_taxon (TaxonDescriptor, optional), criterion, evidence +``` + +Deliberately **not** in `ecological_interactions`: excluded candidates are not members, and +these are not interactions within the community. Making them edges is what produced the +#300 defect. + +`excluded_taxon` must be optional, because ARC is exactly the case where the excluded +strains cannot be distinguished from the retained ones at the available resolution. A +counter-selection with a criterion and evidence but no resolved taxon is still far more +useful than prose. + +**Open question for the curator:** should excluded candidates ever appear in `taxonomy`? +Recommendation: **no** — they are not members, and adding them would re-create #304-style +connectivity noise. + +--- + +## 5. Recommended sequence + +1. **#304 change (1)** — credit `COMMUNITY_LEVEL`. Smallest, no data migration, unblocks + #273, and removes an active perverse incentive. +2. **#294** — status enum plus mechanical backfill. Self-contained, and retires the + recurring misreading of GTDB coverage as GTDB backlog. +3. **#307** — the largest, and the one most worth designing carefully rather than quickly, + since it introduces a concept the schema does not yet have. + +Note there is **no** independent quick win here: the five apparently-groundable taxa in +§3.1 are the withheld ones, blocked on #292 (correcting two NCBITaxon ids that name the +wrong organism). Fixing #292 is worth doing on its own merits and would let those two ground +themselves on the next tool run. + +## 6. What is deliberately not proposed + +- **A general absence-annotation framework.** Three instances is not enough evidence to + design one, and §2 argues they are not the same kind of problem. +- **A `NOT_APPLICABLE` value anywhere.** It reintroduces the ambiguity this proposal is + trying to remove; `NO_GTDB_EQUIVALENT` says why, and "not applicable" does not. +- **Backfilling `abundance_level`.** The perverse incentive in §3.2 should be removed by + fixing the audit, never by filling the slot. `AbundanceEnum` is quantitative and most + sources do not report abundances. From e167646bfde572d875aa01e70b739c7f8d82e070 Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Mon, 3 Aug 2026 10:13:23 -0700 Subject: [PATCH 2/3] Simulate the #304 rule change; the result reverses the recommendation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of this proposal found §4.2 asserting rather than measuring. It claimed "doing (1) without (2) is safe and strictly an improvement". Simulating all four configurations over the current KB shows that is wrong: today — no credit, exemption kept 32 drop the exemption only 390 credit COMMUNITY_LEVEL only 1 credit COMMUNITY_LEVEL + drop the exemption 19 Crediting COMMUNITY_LEVEL alone is indeed safe, but it drops reporting to ONE finding across 307 records. The rule becomes vacuous, and a gate that never fires is not an improvement over one that fires for the wrong reason. Dropping the exemption alone reports 390 — a 12x increase that buries the signal. Only together do they land somewhere defensible: 19, fewer than today's 32 while measuring what the name promises, and a real work-list rather than an artefact. §4.2 now carries the table and says the halves must ship together; §5's sequence is updated from "change (1) first" to "both changes together, do not split". This is the second time in this proposal that checking a number changed the recommendation — the first was discovering the 5 apparently-groundable taxa were deliberately withheld. Both were claims that looked safe enough to assert. Co-Authored-By: Claude Fable 5 --- proposals/absence_semantics.md | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/proposals/absence_semantics.md b/proposals/absence_semantics.md index fb228e99d..f25c12647 100644 --- a/proposals/absence_semantics.md +++ b/proposals/absence_semantics.md @@ -150,17 +150,28 @@ what the tool computes, so it cannot drift — the shape used by `tests/test_enu ### 4.2 #304 — fix the auditor, change no data *(code, small)* -Two independent changes: +Two changes, which must be made **together**. Simulated over the current KB: + +| configuration | `DISCONNECTED` reported | +|---|---:| +| today — no credit, exemption kept | 32 | +| drop the exemption only | **390** | +| credit `COMMUNITY_LEVEL` only | **1** | +| **credit `COMMUNITY_LEVEL` + drop the exemption** | **19** | 1. **Credit `COMMUNITY_LEVEL` interactions.** Treat every taxon in a record as connected by - a community-level interaction, which is what such an interaction asserts. This alone - removes the structural penalty on 107 records. -2. **Drop the `abundance_level`/`functional_role` exemption.** With (1) in place, the - exemption's original purpose — suppressing false positives on records that describe - membership without pairwise edges — is served properly rather than by proxy. + a community-level interaction, which is what such an interaction asserts. +2. **Drop the `abundance_level`/`functional_role` exemption**, so the rule measures + connectivity rather than slot-completeness. + +The table overturns the obvious plan of doing (1) first as a safe increment. It *is* safe, +but it drops reporting to **1 finding across the whole KB** — the rule becomes effectively +vacuous, and a gate that never fires is not an improvement over one that fires for the wrong +reason. Conversely (2) alone reports 390, a 12× increase that buries the signal. -Doing (1) without (2) is safe and strictly an improvement. Doing (2) without (1) would -surface a large backlog at once and should not be done alone. +Only together do they give a defensible number: **19**, fewer than today's 32 while actually +measuring the thing the name promises. That 19 is also a real work-list rather than an +artefact — it is the set of taxa with no interaction of any kind. **This is prerequisite for #273.** That issue cannot decide whether to restore the network gate while `DISCONNECTED` means something other than its name. @@ -192,8 +203,9 @@ connectivity noise. ## 5. Recommended sequence -1. **#304 change (1)** — credit `COMMUNITY_LEVEL`. Smallest, no data migration, unblocks - #273, and removes an active perverse incentive. +1. **#304, both changes together** — credit `COMMUNITY_LEVEL` *and* drop the exemption. No + data migration, unblocks #273, removes an active perverse incentive, and lands on 19 + findings. Do not ship the halves separately: one is vacuous, the other is noise. 2. **#294** — status enum plus mechanical backfill. Self-contained, and retires the recurring misreading of GTDB coverage as GTDB backlog. 3. **#307** — the largest, and the one most worth designing carefully rather than quickly, From 52abc8b60afed1dc1827806e5dc280bf6a77c59d Mon Sep 17 00:00:00 2001 From: "marcin p. joachimiak" <4625870+realmarcin@users.noreply.github.com> Date: Mon, 3 Aug 2026 14:23:43 -0700 Subject: [PATCH 3/3] Refresh the proposal against reality: #304 shipped, and re-measuring found a gap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of this PR re-derived its own figures against current main rather than trusting them, which changed three things. **§4.2 is no longer a proposal.** #304 shipped as PR #311 while this sat open — both halves together, landing at exactly the predicted 19 findings, network issues 49 -> 23. It is marked IMPLEMENTED and kept as the worked example: it is the only one of the three where the simulation can be checked against the outcome. Two things it surfaced are cross-referenced (#312 all-or-nothing credit, #313 dangling-edge detection living in an orphaned script). **§3.1's numbers moved, and re-checking them found a real gap.** Refreshed to 311 records / 1024 taxa: 288 no-equivalent, 87 ambiguous, 6 groundable. Five of the six are the withheld entries from #292 as before — but the sixth is a genuine gap (#314): a taxon whose term.id was corrected after grounding ran, leaving the derived block absent. That incident is the proposal's own thesis in miniature, so it is written up as such. One blank column now encodes four distinct situations — impossible, undecidable, deliberately withheld, accidentally stale — and telling them apart required running the grounding tool over the whole KB and cross-referencing the withheld list. With the proposed enum it would be a one-line query. **Stale figures corrected**: engineering_design 193 -> 197 records, the 92% exemption stated in past tense now that it no longer exists, and the decision line narrowed from three remedies to the two still open. #314 also notes a fix worth doing regardless of what is decided here: a gtdb_classification records ncbi_source_id, so a mismatch against term.id is detectable today with a cheap test and no schema change. Co-Authored-By: Claude Fable 5 --- proposals/absence_semantics.md | 113 +++++++++++++++++---------------- 1 file changed, 57 insertions(+), 56 deletions(-) diff --git a/proposals/absence_semantics.md b/proposals/absence_semantics.md index f25c12647..bfb7c5621 100644 --- a/proposals/absence_semantics.md +++ b/proposals/absence_semantics.md @@ -4,8 +4,9 @@ Three open issues turn out to be one diagnosis. This proposal states it, quantif instance, and recommends a remedy per instance — which is **not** the same remedy three times. -**Decision requested:** approve or reject each of the three remedies in §4. They are -independent; approving one does not commit to the others. +**Decision requested:** approve or reject the two remaining remedies in §4 — the enum in +§4.1 and the counter-selection block in §4.3. They are independent of each other. The third +(§4.2) has since been implemented and is kept here as the worked example. --- @@ -49,33 +50,34 @@ benefit. They are proposed separately below and can be done in any order. ## 3. The three instances, with numbers -### 3.1 #294 — GTDB grounding: 372 blanks, none of which are pending work +### 3.1 #294 — GTDB grounding: 378 blanks encoding four different situations -Measured 2026-08-03 over 307 records / 1007 taxonomy entries: +Re-measured over **311 records / 1024 taxonomy entries** (figures refreshed after PRs +#308 and #311 landed): | | count | share of ungrounded | |---|---:|---:| -| grounded | 635 of 1007 (63%) | — | -| **no GTDB equivalent** — eukaryote, virus, environmental pseudo-taxon, absent from mapping | 285 | 76% | -| **ambiguous** — GTDB splits the NCBI taxon with no majority | 85 | 23% | -| **groundable by the tool** | 5 occurrences | 1% | - -Checking what those last five are is what makes the case. Every one is an entry -**deliberately withheld** under #292 — *Bacteroides ovatus* on `NCBITaxon:821` -(*Phocaeicola vulgatus*) and `Nitrospiraceae bacterium` on `NCBITaxon:1236` -(Gammaproteobacteria), whose ids name a different organism, pinned by -`tests/test_gtdb_withheld_groundings.py` so a tool re-run cannot reinstate them. - -So the honest tally is **370 permanently ungroundable, 2 deliberately withheld, and zero -pending work** — three distinct states rendered as one blank. GTDB grounding is complete to -its ceiling and the schema cannot say so. This is exactly why #276 read as ~40% outstanding -work when the achievable ceiling is ~63%: the issue — which I wrote — mistook impossibility -for backlog, and nothing in the data could have corrected that reading. +| grounded | 646 of 1024 (63%) | — | +| **no GTDB equivalent** — eukaryote, virus, environmental pseudo-taxon, absent from mapping | 288 | 76% | +| **ambiguous** — GTDB splits the NCBI taxon with no majority | 87 | 23% | +| **groundable by the tool** | 6 occurrences | 2% | + +Checking what those last six are is what makes the case, and it took running the grounding +tool across the whole KB and cross-referencing to find out. Five are entries **deliberately +withheld** under #292 — *Bacteroides ovatus* on `NCBITaxon:821` and `Nitrospiraceae +bacterium` on `NCBITaxon:1236`, whose ids name a different organism, pinned by +`tests/test_gtdb_withheld_groundings.py`. The sixth is a **genuine gap** (#314): a taxon +whose `term.id` was corrected *after* grounding ran, leaving the derived block absent. + +So one blank column encodes four different situations — permanently impossible, +undecidable, deliberately withheld, and accidentally stale — and distinguishing them +required re-deriving the whole computation. That is the argument in miniature. +It also means #294 has a **fourth** state to represent, distinct from the other three: It also means #294 has a **fourth** state to represent, distinct from the other three: grounding withheld pending an upstream correction. -### 3.2 #304 — DISCONNECTED fires on the wrong criterion +### 3.2 #304 — DISCONNECTED fired on the wrong criterion *(now fixed — PR #311)* Two rules interact in `network/auditor.py`: @@ -83,8 +85,8 @@ Two rules interact in `network/auditor.py`: `COMMUNITY_LEVEL` interaction has neither by design, so it contributes **no** connections. **107 of 302 records with interactions (35%) are entirely `COMMUNITY_LEVEL`** — structurally, a third of the KB has zero connected taxa. -2. A taxon carrying `abundance_level` **or** `functional_role` is exempt (line ~258). - **931 of 1007 taxa (92%) are exempt on this basis.** +2. A taxon carrying `abundance_level` **or** `functional_role` was exempt. + **931 of 1007 taxa (92%) were exempt on this basis** at the time of measurement. The exemption is doing essentially all the work. `DISCONNECTED` does not mean "this taxon has no curated interaction"; it means "this taxon has no *pairwise* interaction **and** no @@ -111,9 +113,9 @@ There were two places to put that, and both are bad: - **As `engineering_design.notes` prose** — accurate but invisible to any query. PR #305 chose prose. That was right for the record and means the fact is no longer -queryable. Screening is how most SynComs are built, so this recurs: 193 records carry -`engineering_design`, and only 40 carry free-text `notes` — the rest have no natural place -for a negative result at all. +queryable. Screening is how most SynComs are built, so this recurs: **197 records** carry +`engineering_design` and only **40** carry free-text `notes` — the rest have no natural +place for a negative result at all. --- @@ -148,33 +150,31 @@ what the tool computes, so it cannot drift — the shape used by `tests/test_enu **Cost:** one schema change, one datamodel regeneration, one backfill run, one test. **Benefit:** ~63% coverage stops looking like ~63% completion. -### 4.2 #304 — fix the auditor, change no data *(code, small)* +### 4.2 #304 — fix the auditor, change no data — **IMPLEMENTED (PR #311)** -Two changes, which must be made **together**. Simulated over the current KB: +Shipped as proposed: `COMMUNITY_LEVEL` interactions now credit their members, and the +`abundance_level`/`functional_role` exemption is removed. Both halves together, as the +simulation required. | configuration | `DISCONNECTED` reported | |---|---:| -| today — no credit, exemption kept | 32 | -| drop the exemption only | **390** | -| credit `COMMUNITY_LEVEL` only | **1** | -| **credit `COMMUNITY_LEVEL` + drop the exemption** | **19** | - -1. **Credit `COMMUNITY_LEVEL` interactions.** Treat every taxon in a record as connected by - a community-level interaction, which is what such an interaction asserts. -2. **Drop the `abundance_level`/`functional_role` exemption**, so the rule measures - connectivity rather than slot-completeness. +| before | 32 | +| drop the exemption only | 390 | +| credit `COMMUNITY_LEVEL` only | 1 | +| **both — shipped** | **19** | -The table overturns the obvious plan of doing (1) first as a safe increment. It *is* safe, -but it drops reporting to **1 finding across the whole KB** — the rule becomes effectively -vacuous, and a gate that never fires is not an improvement over one that fires for the wrong -reason. Conversely (2) alone reports 390, a 12× increase that buries the signal. +The prediction held exactly: network issues went 49 → 23, of which `DISCONNECTED` 32 → 19, +and the surviving findings are genuine (spot-checked a *"Desulfovibrio piger comparator"* +and *"Sporomusa spp. in KB-1"*, both in `taxonomy` while taking part in no interaction of +any scope). -Only together do they give a defensible number: **19**, fewer than today's 32 while actually -measuring the thing the name promises. That 19 is also a real work-list rather than an -artefact — it is the set of taxa with no interaction of any kind. +Two things surfaced during that work and are tracked separately: the credit is +all-or-nothing because the schema cannot say *which* members a community-level interaction +concerns (#312, which overlaps §4.3's design question), and dangling-edge detection turns +out to live only in an orphaned script that nothing runs (#313). -**This is prerequisite for #273.** That issue cannot decide whether to restore the network -gate while `DISCONNECTED` means something other than its name. +**This retired the blocker on #273**, which could not decide whether to restore the network +gate while `DISCONNECTED` measured something other than its name. ### 4.3 #307 — add a counter-selection block *(schema, medium)* @@ -203,18 +203,19 @@ connectivity noise. ## 5. Recommended sequence -1. **#304, both changes together** — credit `COMMUNITY_LEVEL` *and* drop the exemption. No - data migration, unblocks #273, removes an active perverse incentive, and lands on 19 - findings. Do not ship the halves separately: one is vacuous, the other is noise. -2. **#294** — status enum plus mechanical backfill. Self-contained, and retires the - recurring misreading of GTDB coverage as GTDB backlog. +1. ~~**#304, both changes together**~~ — **done, PR #311.** Landed at 19 findings and + retired the blocker on #273. +2. **#294** — status enum plus mechanical backfill. Now the next step. Self-contained, and + it retires the recurring misreading of GTDB coverage as GTDB backlog. §3.1 shows the + distinction currently costs a full re-derivation to recover. 3. **#307** — the largest, and the one most worth designing carefully rather than quickly, - since it introduces a concept the schema does not yet have. + since it introduces a concept the schema does not yet have. Worth designing together with + **#312**, which asks the same question from the other direction: how to say *which* + members a community-level statement concerns. -Note there is **no** independent quick win here: the five apparently-groundable taxa in -§3.1 are the withheld ones, blocked on #292 (correcting two NCBITaxon ids that name the -wrong organism). Fixing #292 is worth doing on its own merits and would let those two ground -themselves on the next tool run. +Independent of all three: **#314** (a taxon whose id changed after grounding, leaving a +stale blank) is a one-command data fix, and the `ncbi_source_id` mismatch it exposes is +detectable with a cheap test today — no schema change required. ## 6. What is deliberately not proposed