Skip to content

v8.0.0 — Level 6: honest predictor judge + reality filter

Choose a tag to compare

View all tags
@erlanders177 erlanders177 released this 23 Jul 09:46
· 43 commits to main since this release
v8.0.0
a1d1f39

The biggest update in BioForge's history. Two judgment tools (not predictors)
and a safety net against silent data corruption.

What's new

Level 6 — the honest judge (EvolutionBenchmark)

Judge any evolution predictor honestly. Every check exists because it caught us:

  • Trivial-baseline bar — the rival is not 0.5, it's the best free axis (frequency,
    site mutability, conservation). Our own "AUC 0.80" turned out to be the mutability axis.
  • Novel-mutation regime — separates already-circulating mutations (where counting is
    enough) from genuinely new ones.
  • Bootstrap 95% CI — small wins that evaporate on resampling are exposed.
  • Pretraining-leakage detector — flags a model that remembers instead of predicting.

Judged on real H3N2 flu: our trained model scores AUC 0.837 global / 0.631 on novel
mutations, over the trivial mutability bar (0.793).

Level 6 — the reality filter (RealityCheck)

Of the "concerning" mutations another tool (EVEscape, ESM-2, a DMS assay) hands you,
which ones have real traction in the population? Plugs in behind any of them.
Two tiers, never mixed: OBSERVED (evidence, AUC 0.97) vs ESTIMATED (conjecture, AUC 0.72).

Data-integrity safety net

  • An anti-corruption guard that refuses to mis-encode a mistyped sequence
    (raises instead of silently turning it into N).
  • Property-based invariants over both DNA and protein alphabets.
  • tools/integrity_check.py — a reusable integrity certificate.

Critical bug fixed

The multiple-sequence aligner was packing every protein as DNA, silently turning
non-ACGT residues into N. It corrupted the whole Level 5 foundation. Fixed, model
retrained on clean data, every number re-measured. The two new Level 6 tools are
what caught it.

Requirements and install

  • Python >= 3.10
  • Only dependency: NumPy (the C engine and the trained model ship pre-compiled)
pip install bioforge
Optional deep-learning axis (ESM-2), not needed for the core:


pip install "bioforge[ai]"
Tests
498 tests passing — including property-based integrity tests over both alphabets
and the full Level 6 judge/filter battery.

Command-line tools
bioforge-analyze · bioforge-qc · bioforge-bgzip · bioforge-evolution
Assets 2
Loading