Skip to content

v0.7.5

Latest

Choose a tag to compare

@lucapinello lucapinello released this 18 Aug 13:15
· 9 commits to main since this release
cc8aabe

[0.7.5] — 2026-08-18

Fixed

  • predict_variant_effect(..., assay_ids=None) no longer fails on alphagenome_pt. It raised
    invalid literal for int() with base 10: 'logits'; the default is all 5,168 tracks and the 4
    SPLICE_SITES tracks are always among them, so every default-argument call broke while explicit ids
    worked. Head outputs are dicts keyed by two unrelated things — resolution ({1: ..., 128: ...}) for
    most heads, tensor kind ({"logits", "probs"}) for the splice-site classification head — and the
    extraction loop overwrote its resolution variable with whichever key it picked before calling int()
    on it. Selecting a tensor and reporting a resolution are now separate operations.

  • The splice-site tracks were being read as logits rather than probabilities. Fixing the crash
    exposed it: the old fallback took the first dict key, which is logits by insertion order. The JAX
    reference returns {'logits', 'predictions'} and treats the softmax as the prediction, so the two
    AlphaGenome backends would have disagreed on those 4 tracks — unbounded logits against values in
    [0, 1]. Affects SPLICE_SITES/{donor,acceptor}/{+,-} on alphagenome_pt only.

  • The env-mode subprocess template held a second copy of the same logic, so the first fix left the
    default path (use_environment=True) broken while every unit test passed. The template now imports
    the shared helpers, with source-level guards that fail if it ever restates them.

  • 734 SPLICE_SITE_USAGE tracks were returning log-space values on alphagenome_pt. The fix above
    covered the splice-site classification head, whose activated tensor the port calls probs. The
    usage head calls the same thing predictions (matching the JAX reference) and also carries a
    track_mask, so its 734 tracks kept falling through to raw logits: PyTorch emitted -13.8..-11.3
    where JAX gives 1e-6..1.2e-5, with exp(logits) recovering the JAX values to 2.7%. Across the full
    1 MB SORT1 window the two backends correlated at 0.20 for this head.

    This mattered beyond the raw numbers. alphagenome_pt has no background null of its own —
    _CDF_ALIASES maps it to alphagenome — so those log-space values were being ranked against a
    sigmoid-space null, making percentiles for the affected tracks meaningless rather than merely noisy.
    alphagenome_pt is in _MULTI_TRACK_ORACLES, so a discovery run enumerates all of them. After the
    fix the two backends agree at correlation 1.0000 and 4.9% peak-relative difference, in line with every
    other head. No committed artefact is affected: the two example outputs containing
    SPLICE_SITE_USAGE were both produced by the JAX backend, so nothing needs regenerating.

    Tensor selection now tries ("predictions", "probs") in order rather than one hardcoded name, never
    falls back to logits or to a *_mask entry while any activated key exists, and warns loudly when it
    cannot find one — a silent fall-through to logits is invisible downstream, since log-space values are
    still finite, still float32, and still the right shape.

Testing

  • The backend-equivalence test now covers every output type, not three DNASE tracks. That test is
    what makes the _CDF_ALIASES premise ("the two backends produce identical predictions") checkable, and
    its three tracks all came from one head, at 1 bp, returning a bare tensor — the easiest case in the
    model. Both divergences above lived in dict-keyed heads it never touched. Bounds come from full-array
    measurement (correlation > 0.99, peak-relative < 8%, versus a worst measured head of 4.945%); the
    existing 2% bound on the three DNASE tracks is left untouched rather than loosened, so coverage grows
    without trading away sensitivity where it is already earned.

Documentation

  • README audit, section by section. Every CLI invocation, flag, Python API call, track count,
    sequence length and stated size was checked against the code or the shipped artefacts. Most of it was
    already right — the coordinate-convention table, the Cherimoya 1,149/369 assay split, AlphaGenome's
    1 bp vs 128 bp bin sizes, the 24 MCP tools, the 4 K562 ATAC experiments and every sequence length all
    reproduce exactly. Five things did not:

    • The install instruction said git checkout v0.7.4 and the citation BibTeX said 0.7.3, two
      releases stale. Both are copy-paste targets: one decides which code a reader installs, the other what
      they cite in a paper. Now guarded against chorus.__version__, which nothing had tied them to.
    • The disk breakdown understated backgrounds by ~1.5 GB (~1.9 → ~3.4 GB) and its total with it
      (~85 → ~87 GB), because the 0.7.4 Sei rebuild grew that NPZ from 40 sequence classes to all 21,947
      tracks. The existing guard summed the table, which stayed self-consistent while being wrong, so the
      new guard compares the bucket against the shipped NPZs.
    • The AlphaGenome backend-equivalence claim was understated and, for one assay, false. It said
      "within 1–2 % relative error across all 7 chorus-exposed assays"; measured per head it is 0.8–4.9 %
      at correlation 0.9998–1.0000, and the SPLICE_SITES assay — one of those 7 — was not equivalent at
      all before this release. Now states the measured range, points at the test that enforces it, and tells
      anyone who ran alphagenome_pt on splice tracks to re-run.
    • perbin_cdfs is omitted for three oracles, not two (EPInformer-seq was missing), and
      signed_floor_rescale_batch is a normalizer method rather than an _igv_report function — a reader
      following that sentence looked in the wrong module.
    • The contributing checklist omitted _describe_tracks(), the hook behind the uniform
      describe_tracks(). Leaving it out recreated exactly the friction that method was added to remove.
    • Also: the two tables sizing the background NPZs use binary and decimal units respectively, which read
      as contradicting each other (~260 MB vs 279 MB for the same file). Both now say so.
  • describe_tracks() is now documented in the README. It shipped in 0.7.4 as the one uniform way
    to ask any of the nine oracles what it can predict, and the README's "Discovering tracks" section
    still told users to import an oracle-specific metadata module — the exact friction the method was
    added to remove. The section now leads with describe_tracks() (load-free: no model, no reference
    genome, no GPU) and keeps the per-oracle metadata search as the deeper option. Every snippet is
    copied from real output.

  • The README now states that Enformer predictions vary between processes, with the measured
    numbers (~0.02% typical, 3.4% worst on a raw value, bit-identical within one process), the fact that
    it is the only one of the nine oracles affected, why conclusions are still stable
    (quantile_score ≤0.69%, near_ties_at_cutoff covers changed top-N membership), and the
    TF_DETERMINISTIC_OPS=1 TF_CUDNN_DETERMINISTIC=1 opt-in for bit-exactness. This was previously
    recorded only in the CHANGELOG and the null protocol, so a user who ran Enformer twice and got
    different numbers had nothing in the README to explain it.

  • Cross-process determinism measured for all nine oracles — alphagenome_pt came in bit-exact
    (2/2 pairs) once the crash above stopped blocking it, leaving Enformer as the only oracle that
    drifts between processes
    . Written up in
    audits/2026-08-17_post_v074_focused_audit.md. Corrects two claims in AUDIT_CHECKLIST.md: an
    "AlphaGenome is NOT deterministic" finding that three gate runs now contradict (0.000e+00), and a
    "verified bitwise" claim that held same-process only.

  • TrackRecord.has_background no longer promises more than it delivers: no oracle populates it, so
    the docstring now says so and explains why (describe_tracks() stays download-free), pointing
    callers at NormalizationLoader.has_background.

  • README restructured to cut clutter. It had accumulated 24 blockquote asides totalling 12,229
    characters
    , the longest 1,692 — caveats and per-release detail interrupting the reading path rather
    than living somewhere findable. Now 6,793 characters (−44%), longest 704. The changes are structural,
    not just shorter: one Caveats section holds the four things that can change a number (Enformer
    cross-process variation, the pre-0.7.5 alphagenome_pt splice history, Cherimoya-vs-ChromBPNet
    magnitude, hg38-only), with inline asides reduced to pointers; token plumbing moved from the quick start
    into the installation section; and per-release detail is delegated to this changelog, so the README stops
    growing by a line per release.

  • Second README pass: less lecturing. The first pass moved caveats out of the reading path; this one
    cuts explanation a reader does not act on. Gone: the decimal-vs-binary units lesson on the free-disk
    bullet ("a volume sold as 100 GB is 93.1 GiB, so it fits with ~2 GiB spare"), ~15 lines on why conda envs
    are large (pip hardlinking, du -sc semantics), a units note explaining an inconsistency in our own
    tables, the provenance of an older audit's equivalence numbers, and the history of the background
    downloader's previous fetch rule. Also removed five outright duplications — a comment-only BedGraph code
    block that pointed at the real recipe, two ways to load the same genome, Core concepts re-defining
    "Oracle" and "Track" from Key terms, Key terms re-defining "Conversational genomics" from the
    paragraph three lines above it, and the credentials/conda-env rule stated in two places. Cumulatively
    since 0.7.5: aside text 12,229 → 6,409 chars (−48%), longest aside 1,692 → 704, prose lines over 420
    chars 17 → 8.

    One guard was deleted rather than satisfied: test_the_tldr_install_size_agrees_with_the_disk_table
    existed to keep the TLDR's install size in sync with the disk table, and the TLDR no longer states an
    install size, so there was nothing left to keep in sync.

  • Six documentation defects fixed, found auditing notebooks, examples, docs and the nulls:

    • examples/walkthroughs/README.md described Sei by its pre-0.7.4 scope ("40 sequence classes") in a
      table the README calls a "full side-by-side comparison" — which also covered only 6 of 8 oracles.
      Corrected, with Cherimoya and EPInformer-seq added.
    • docs/variant_analysis_framework.md still carried the "1–2 % fp32 noise" backend-equivalence claim
      that 0.7.5 corrected in the README: understated, and false for the SPLICE_SITES assay.
    • describe_tracks() appeared zero times in docs/, including the "Full Python API reference". Now
      documented there with its signature, its load-free guarantees, and what it replaces.
    • docs/BACKGROUND_NULL_PROTOCOL.md had no record that alphagenome_pt has no null of its own and
      ranks against alphagenome's. New §8b states the alias, the premise, the test that enforces it, and
      that the test must be extended before another oracle is aliased. Decision logged for 2026-08-18.
    • examples/notebooks/README.md said each notebook "produces … outputs inline" while 16 code cells
      across three ship blank
      — the cells needing a second oracle env or coolbox. Now stated.
    • The free-disk bullet still said a single-oracle install is "~13 GiB, not 85" after the total moved
      to 87.
  • Full findings, what was verified clean, and the guard/false-positive notes:
    audits/2026-08-18_post_v075_docs_and_readme_audit.md.