You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fixed FastMCP default tool leakage in scripts/local_mcp_server.py: wrapped the 6 hosted-overlap fallback tools (search_pubmed, get_pubmed_article, search_biorxiv, search_chembl, search_opentargets, search_clinical_trials) and search_cosmic behind BIONEXUS_LOCAL_HOSTED_FALLBACKS=1. By default, FastMCP now registers exactly 9 local unique tools (GTEx, GEO, STRING, UniProt, Ensembl, gnomAD, PDB, AlphaFold, Reactome) plus 6 Resources and 6 Prompts.
Eliminated Agent routing ambiguity: AI coding agents querying literature, targets, or clinical trials will cleanly route to dedicated cloud-hosted MCP endpoints without duplicate tool confusion.
Synced test coverage: updated tests/unit/test_mcp_server.py to assert 9 default unique tools and 16 tools upon opt-in.
🎖️ Added (Capability Certification Program — BNS-010)
src/bionexus/certification.py: 14 evidence criteria and four tiers (CERTIFIED / VALIDATED / EXPERIMENTAL / CONNECTOR-ONLY). Tiers are computed from recorded evidence, never asserted (BNS-CF-002); structural cross-checks re-verify contract-derived criteria against the live ABI, preconditions, and taxonomy. Honest current state: 0 CERTIFIED, 7 VALIDATED, 1 EXPERIMENTAL — the per-capability blocking-criteria list is the published roadmap to the M4 target of 10 CERTIFIED (evidence must be produced, criteria never weakened, BNS-CF-006). CLI: bionexus certification.
New spec spec/BNS-010-capability-certification.md; tests tests/unit/test_certification.py (CERTIFIED requires all 14 — structurally un-gameable).
🧯 Added (Scientific Failure Taxonomy — BNS-011)
src/bionexus/failures.py: twelve normative failure modes (BN-F001 assay-state confusion, BN-F002 pseudoreplication, BN-F003 unsupported annotation, BN-F004 identifier mismatch, BN-F005 missing multiple-testing correction, BN-F006 invalid model assumption, BN-F007 parameter instability, BN-F008 cross-database contradiction, BN-F009 missing spatial provenance, BN-F010 backend degradation masquerading, BN-F011 claim inflation, BN-F012 unexecuted maturity claim). Each record: definition, canonical example, affected capabilities, detection rule, fail-closed required behavior, acceptable degradation, benchmark cases. Three modes are honestly flagged as open gaps (no benchmark coverage yet). classify_violation() tags runtime violations with taxonomy IDs. CLI: bionexus failures list|show.
New spec spec/BNS-011-failure-taxonomy.md; tests verify record shape, vocabulary, and that every benchmark-case reference resolves to a real eval case.
🚫 Added (Fail-Closed Gate — BNS-005 §6)
src/bionexus/failclosed.py: prevent_invalid_run() — the canonical gate implementing knowing when not to compute is a scientific capability: missing evidence → ABSTAIN (request data), invalid input → REFUSE, backend unavailable → DEGRADE WITH DISCLOSURE, assumption violated → BLOCK CLAIM, claim beyond warrant → BLOCK CLAIM, external validation absent → CAP EVIDENCE LEVEL. No row resolves to silent execution. Returns PreventionDecision with failure-mode IDs, remedies, and the underlying routing decision. CLI: bionexus prevent "<query>".
New spec requirements BNS-AD-013..015; tests cover all six rows plus the clean RUN PERMITTED exit.
📒 Added (Claim–Evidence Ledger — BNS-012)
src/bionexus/ledger.py: claims as auditable dependency graphs — ClaimRecord (supported_by / contradicted_by / depends_on) over closed-vocabulary EvidenceRef nodes (dataset, transformation, method_run, statistical_result, database, cross_method). Fail-closed resolution: any contradiction forces CONFLICTED; no support forces ABSTAIN; otherwise the weakest supporting warrant, clamped by the capability's ABI evidence ceiling (database/cross-method support counts as external validation). JSON round-trip + PROV-O JSON-LD projection; append-only (duplicate IDs rejected). Deliberately a data structure, not a graph platform. CLI: bionexus ledger show|jsonld.
New spec spec/BNS-012-claim-evidence-ledger.md; tests include the CLAIM-017 reference scenario.
ABI clamp refinement: warning states (FRAGILE / CONFLICTED / ABSTAIN / UNASSESSED) are never rewritten by evidence ceilings — only ascending-ladder warrant levels (PRELIMINARY→REPLICATED) are clamped.
spec/ normative specification tree: nine RFC 2119-style documents (BNS-001..BNS-009) plus index, defining the scientific contract that binds BioNexus and any connected host agent — capability contract & Scientific ABI, input semantic invariants, execution fidelity, evidence maturity, abstention & degradation, provenance, cross-method validation, host conformance, and capability lifecycle. Every requirement carries a stable ID (BNS-XX-nnn) with a live verification hook (unit test, eval category, or runtime refusal). tests/unit/test_spec_conformance.py enforces document presence, RFC 2119 keyword usage, and cross-document reference integrity.
🧬 Added (Biological Capability ABI — bionexus.abi, ABI v1.0)
Capability contracts upgraded from metadata to a Scientific ABI: every canonical capability now projects to a machine-readable ABI record (input_contract with allowed matrix states and coordinate types, preconditions, forbidden_claims, execution reference backend/algorithm, validation policy, evidence_ceiling, provenance requirements), generated from the canonical contract so it cannot drift. forbidden_claims and evidence_ceiling_without_external_validation are new normative fields on CapabilityContract.
Normative forbidden-claim taxonomy (FORBIDDEN_CLAIM_CATALOG): 11 claim families (causal interaction, cell-cell communication, cell-type identity without reference, clinical diagnosis, treatment recommendation, model substitution, hazard causation, true-expression recovery, sensor calibration, regulatory compliance, pipeline results without execution) with detection patterns.
Routing-time forbidden-claim interception (BNS-AD-009): requests asking a capability for a claim on its forbidden list are now deterministically refused with the scientific reason and reformulation remedy (e.g. "use Moran's I to prove cell-cell communication" → ABSTAIN).
Evidence-ceiling clamping (enforce_evidence_ceiling): over-warranted maturity claims are clamped to the capability's ceiling (spatial SVG → FRAGILE without external validation; exploratory clustering → PRELIMINARY; REPLICATED requires external truth sets).
CLI: bionexus abi list|show <id>|audit-claims <id> --claims ...|conformance.
New unit tests: tests/unit/test_abi.py (10 tests: projection completeness, single-source-of-truth, claim audits, ceiling clamps, router interception, no-false-positive controls, CLI surface).
Frontier calibration track: new evals/datasets/calibration_edge.yaml (11 probes, known_limitation: true) exploring adjacent-rank maturity discrimination, coordinate-substitution detection, statistical-power auditing, multi-intent routing, and ABI ceiling clamps. Frontier cases are executed and reported with honest pass/fail but excluded from gating metrics until graduation.
Honest benchmark reporting: reports now separate gating (guaranteed behavior) from frontier (known limitations) and state the union accuracy — gating-only 100% is explicitly labeled NOT a calibration claim. Current honest state: gating 42/42 (CRI 100%), frontier 7/11, union 49/53 = 92.5%, union calibration verdict UNDERCONFIDENT (macro-F1 96.3%, OCE 0.041). The four open known limitations are published by name in every report.
Calibration metrics extended: adjacent-rank error rate, within-one accuracy, per-class precision/recall/F1, calibration verdict, skipped-no-backend accounting, and cross-host consistency (single-host runs reported as not evaluated rather than trivially consistent).
L2/L3 maturity attribution made honest: claim audits warrant at most PRELIMINARY (they verify absence of overclaim, not statistical support); L3 outcome cases attest SUPPORTED only when the gold pipeline actually recovered the planted signal, and are excluded from calibration (disclosed count) when optional backends are absent. Fixes the previously un-reproducible "100% calibration" committed report (the prior numbers masked 12 L2/L3 maturity mismatches).
Spec-strengthened gating cases (BNS-AD-009): claim-gxppart11-001 and claim-acmg-clinical-001 now expect ABSTAIN — requesting an FDA Part 11 certified audit trail or an official clinical diagnostic report is refused at routing time instead of being permitted and audited post-hoc. Three new forbidden-claim refusal cases added (refuse-forbidden-*).
Loader fix: expected_maturity is now actually read from YAML suites (was silently dropped).
New unit tests: tests/unit/test_calibration_frontier.py (8 tests) and updated test_eval_harness.py.
🔒 Fixed (Single source of truth for the plugin mirror trees)
Dual skill-tree drift eliminated: skills/single-cell-rna-qc/scripts/scrna_pipeline.py had diverged by 49 lines between the canonical root tree and the plugins/bionexus/skills/ copy (the mirror lacked --run-dir / Run Capsule support). The mirror is now regenerated from the root; both trees are byte-identical.
Drift detection now covers code trees, not just JSON manifests: bionexus registry --check and scripts/registry_compiler.py --check verify skills/ and scripts/ are byte-identical to their plugins/bionexus/ mirrors (content edits, missing files, and stale mirror-only files all fail CI); --generate resynchronizes the mirrors automatically. Ignored artifacts (__pycache__, logs, doctor cache) are excluded.
Rule: edit only the canonical root skills/ and scripts/ trees — the plugins/bionexus/ copies are compiler-generated.
Removed the broken plugins/codex/ scaffold: its only manifest (.codex-plugin/plugin.json) declared "skills": "./skills/" pointing at a directory that never existed there; nothing in the repo referenced it.
Completed the self-contained plugins/bionexus/ plugin root: added the missing plugins/bionexus/.mcp.json required by its .codex-plugin/plugin.json ("mcpServers": "./.mcp.json"), so every relative reference in the nested manifests now resolves.
New unit tests: repository-level mirror zero-drift, all three divergence classes (edited / missing / stale) plus sync repair, and self-containedness of the plugin-root manifests.
L3 auto-pass defect: evals/runner.py no longer records PERMITTED when a scientific backend (scanpy/squidpy/pydeseq2/...) is unavailable. Missing backends now produce SKIPPED_NO_BACKEND results that are never counted as passed, are excluded from the accuracy denominator in non-strict runs, and are listed in a dedicated report section with the unavailable-backend reason. A machine without backends can no longer produce a passing L3 score.
Strict mode: new bionexus eval --strict flag (or BIONEXUS_EVAL_STRICT=1) promotes backend-unavailable skips to failures with a non-zero exit code — required when citing an L3 score. New CI job eval-strict installs the full gold-chain + spatial stack (no || true degradation), asserts the backends import, and runs the benchmark in strict mode, uploading the generated report as an artifact.
Unknown L3 planted signals now fail loudly (OUTCOME_MISMATCH) instead of silently auto-passing, so dataset typos cannot inflate scores.
Circular test removed: test_eval_harness.py no longer asserts "benchmark passes with CRI ≥ 0.95" by construction; it now verifies accounting integrity (passed/failed/skip exclusivity, skip reasons, denominator math), plus new regression tests for missing-backend skips, strict-mode promotion, and the --strict CLI flag.
L2 replay disclaimer: offline replay reports now state explicitly that L2 scores audit scripted fixture responses, not live host-agent behavior.
📝 Fixed (Documentation vs SSOT honesty)
README skill table aligned to bionexus.registry.yaml: variant-interpretation B→C, protein-structure-analysis A→C, biologics-design A/C→C, clinical-cohort A/C→C, provenance A→B; backend strings updated to SSOT wording. New guard test tests/unit/test_readme_consistency.py rejects grade overclaims, unverifiable static score badges, and "Primary Active" Python versions absent from the CI matrix.
Python 3.13 claim downgraded to "Experimental — not in CI" (matrix covers 3.10–3.12); host-agent platforms downgraded from "Verified (Tier 1)" to "Declared (not CI-verified)" with an honesty note.
Doctor example output replaced with the real bionexus doctor format; MCP tool names corrected (search_gtex→get_gene_expression, search_pmc→get_pubmed_article); static Tests-217/CRI 96.2% badges replaced with non-numeric equivalents.
Corrected the 0.8.0 entry below: the eval suite is 8 datasets / 39 cases (not 6 datasets / 29 prompts).
