Skip to content

v0.8.0

Latest

Choose a tag to compare

@lorenzoruggerii lorenzoruggerii released this 16 Nov 16:17
aa81327

No percentiles change in this release. No background, region set, retention rule or default
fold moved, so every percentile is the same as in 0.7.5. The minor version is for new surface:
conservation tracks, an annotation catalog, three MCP tools and two CLI commands (#246). It also
covers MCP behaviour changes (#251). discover_variant_cell_types now refuses non-AlphaGenome
oracles instead of returning an empty ranking. A repeat load_oracle that asks for a different
model now raises instead of answering already_loaded. And AlphaGenome track search no longer
returns its 563 padding rows as tracks.

Upgrade if you build environments fresh. Since alphagenome 2.0.0 (2026-11-11), a
chorus-alphagenome env built from v0.7.5's env file fails at predict(). This release pins both
AlphaGenome backends' dependencies (#250, #253), and its imports work under alphagenome 2.0 as well
(#249).

Background artefacts. Unchanged: this release is verified against dataset revision
backgrounds-2026-08-16-sei-full, the same as 0.7.4 and 0.7.5, and _HF_REVISION still pins it.
Nothing was rebuilt or re-uploaded.

Added

  • Conservation-score tracks for IGV reports. chorus/analysis/conservation.py wraps three
    hg38 conservation sources as bulk-downloaded genome-wide bigwigs (mirroring how oracle weights
    and per-track backgrounds are cached): GPN-Star entropy, vertebrate-alignment model (GPN-Star
    also ships mammalian and primate models, not used here), shown as a coverage track plus a
    per-nucleotide sequence-logo track, both displaying clip(1 - entropy, 0, 1) so the most
    conserved positions are tallest/highest; UCSC PhyloP 100-way; and UCSC PhastCons 100-way (both
    raw coverage, the same 100-way vertebrate alignment as the GPN-Star model).
    analyze_variant_multilayer(..., show_conservation=True) adds all of these to the IGV browser,
    capped to a bounded window around the variant. A new chorus conservation CLI subcommand
    lists/downloads the tracks ahead of time; chorus health surfaces their download status.
  • A unified annotation catalog. chorus/utils/annotation_store.py adds AnnotationStore,
    which lists/describes/downloads across the three previously-separate annotation registries
    (conservation tracks, GENCODE GTF gene annotations, and a new user-editable custom-entry YAML
    file) through one interface, plus new list_annotations / describe_annotation /
    download_annotation MCP tools and a chorus annotation CLI subcommand. Downloaded
    bigwig-format entries have their declared genome build physically verified against the file's
    own chromosome-1 length, raising on a confident mismatch rather than silently scoring the
    wrong genome build.

Fixed — fresh AlphaGenome envs

  • A freshly built chorus-alphagenome env failed at predict().
    environments/chorus-alphagenome.yml installed alphagenome and alphagenome-research from
    git with no ref, so a build after 2026-11-11 took alphagenome 2.0.0, which removed the legacy
    alphagenome.models.* import paths: the model loaded, then every prediction raised
    No module named 'alphagenome.models.dna_output'. Both are now pinned to full commit shas,
    the pair the shipped AlphaGenome backgrounds and examples were produced with
    (alphagenome@3563ebf, alphagenome_research@1e55dcf), and
    tests/test_alphagenome_deps_are_pinned.py fails on an unpinned git dep. Existing envs are
    unaffected. Reported, with a forward-compatible import fix, by
    @starrzy in
    #249 — thanks!
  • The PyTorch AlphaGenome backend had the same unpinned git dependency.
    environments/chorus-alphagenome_pt.yml installed alphagenome-pytorch from git with no ref,
    so a fresh chorus-alphagenome_pt env ran whatever upstream HEAD was that day, which is 10
    commits past the build chorus validated (#239-#241). Those commits include a track-strand fix
    and a junction-head change, and the backend borrows the JAX null through the #241 alias. It
    is now pinned to that commit (473d3d2), with alphagenome==0.7.0 alongside: at that commit
    the [scoring] extra leaves alphagenome unbounded, so pip would otherwise resolve 2.0.0.
    tests/test_alphagenome_deps_are_pinned.py now holds every git dep in every env file to a
    full commit sha.

Fixed — review of the conservation / annotation-store work

  • Conservation tracks were plotted onto non-hg38 reports. Every source chorus wraps is
    hg38, and a bigwig read against another assembly returns values rather than erroring, so an
    mm10 report showed human conservation scores against mouse coordinates with no warning.
    show_conservation=True now raises for a non-hg38 report, checked before any work is done.
  • No coverage rendered as maximum conservation. Bigwig NaN was mapped to 0.0 and then
    inverted to clip(1 - 0, 0, 1) == 1.0, so an assembly gap drew a solid full-height bar
    indistinguishable from a perfectly constrained base. Uncovered positions are now omitted;
    genuine zeros still plot.
  • chorus annotation remove --delete-file deleted files chorus never downloaded. For a
    kind="local" entry the registered path is the user's own file. It is now left alone — a
    test previously asserted the opposite.
  • The GPN-Star downloads were unpinned, fetching songlab/gpn-star-scores at its head, so
    a re-upload could change conservation values silently. Pinned, with a guard that reads the
    track configs rather than the call site.
  • The sequence-logo track was off by one base against every coolbox track above it:
    coolbox passes 0-based half-open ranges, the height function reads 1-based inclusive.
  • A custom_annotations.yaml with an empty annotations: key broke every listing path
    with an AttributeError naming neither the file nor the key — setdefault only fires when
    a key is absent, not when it is present-but-empty.
  • A source with no derivable filename was accepted at registration and failed later with
    AttributeError: 'NoneType' object has no attribute 'exists'; it now raises with advice.
  • A 0-byte annotation file printed the optimistic size estimate instead of its real size —
    the one case a user needs to see — because the check was truthiness, not is not None.
  • The assembly check did not run on the path that reads the data. It was called only from
    describe_annotation/add_annotation, which a report never touches. _bigwig_path now
    verifies after download: a confident mismatch raises, an unreadable file warns.
  • _write_html_report (MCP) could report a path to_html never wrote to. Both re-derived
    output_dir / default_filename() independently; for an output_dir like /data/run.v2 the
    dotted .v2 makes Path.suffix non-empty, so to_html treats the whole path as the file
    while _write_html_report still appended a filename onto it — a path that does not exist.
    VariantReport/CausalResult now expose resolve_html_path, and both to_html and the MCP
    helper call it, so they cannot disagree.
  • discover_variant rendered and wrote its own HTML report twice. It called
    discover_variant_effects(..., output_path=...), which builds and writes the report — then
    stamped analysis_request on afterwards and wrote it again to get the user prompt into the
    file. discover_variant_effects already accepted an analysis_request kwarg for exactly this
    case; the server now builds it up front and passes it in, so the report renders once and the
    server just looks up where it landed. Expensive when show_conservation makes each render
    nontrivial, wasted otherwise.

Fixed — what the MCP server tells an agent

An agent sees only the system prompt, the getting_started prompt, list_oracles' specs and
the tool descriptions. A sweep of that text on 2026-11-15 found it wrong in ways that change
what an agent does, and the sweep turned up behaviour bugs in the same layer.

  • getting_started undercounted and misled. It said "all 7 available oracles", left
    Cherimoya out of its menu, gave AlphaGenome as "5,731 tracks" (the metadata row count; 563
    rows are padding placeholders, so 5,168 are queryable) and described Sei as "not per-track
    signal" (it has 21,907 chromatin profiles with percentiles). The same 5,731 appeared in
    list_oracles' AlphaGenome specs, recommend_alphagenome_backend and three docstrings.
  • Its example track searches matched nothing. 'DNASE K562' and 'CAGE liver' return no
    tracks on any oracle: the query is one substring, and the ids read DNASE:K562. The
    truncation note's advice to "add an assay or cell type" failed the same way. The examples,
    the note and the list_tracks help now explain the query. The note's example is chosen per
    oracle, because 'DNASE:HepG2' finds nothing on Cherimoya or Sei.
  • list_tracks('alphagenome', ...) returned padding rows as tracks: 116 of the 128 rows
    for 'PROCAP', 79 of 384 for 'DNASE'. AlphaGenomeMetadata.search_tracks now skips them.
  • discover_variant_cell_types claimed an Enformer screen it cannot run. Its scout tracks
    are always AlphaGenome's, so any other oracle came back with an empty ranking, which reads
    as "no cell type is affected". It now refuses other oracles and names discover_variant.
  • load_oracle answered already_loaded to a request for a different model, and never
    said which model it held, so loading ChromBPNet K562 after HepG2 kept HepG2 without saying
    so. A conflicting repeat load is now an error that says how to switch. load_oracle and
    oracle_status now report loaded_model, read from the oracle itself, because
    discover_variant reloads per-model oracles in place. ChromBPNet now records model_type,
    so bias-aware versus nobias counts as a different model.
  • load_oracle sent every selection argument to load_pretrained_model. EPInformer-seq
    takes assay only in its constructor, so its two H3K27ac assays (22 of its 33 tracks) were
    unreachable over MCP, and an argument an oracle does not take at all surfaced as a bare
    TypeError. Arguments now go to whichever of the two takes them, and anything else is
    refused by name before any loading starts.
  • LegNet could not switch cell type at load time. load_oracle('legnet', cell_type=...)
    raised TypeError, leaving only HepG2 reachable over MCP. discover_variant hit the same
    TypeError for all three LegNet models and returned nothing. LegNetOracle.load_pretrained_model
    now takes cell_type, as EPInformer-seq's already did.
  • Smaller corrections: Enformer no longer advertises RNA (it has no RNA track); the
    oracle_name help lists alphagenome_pt; the system prompt's "score each variant once" rule
    says which oracles hold one model per load and no longer lists discover_variant_cell_types
    as a multi-variant tool; predict and score_prediction_region give Enformer identifier
    examples ("CAGE:K562" resolved nowhere, and a description such as "DNASE:K562" passes
    validation but is rejected by the env template); AlphaGenome's get_track_summary() sums to
    5,168, not 5,731; conservation
    downloads are 5.5–11.6 GB each, not "7–44 GB"; the multilayer scoring list includes splicing,
    Sei classes and EPInformer-seq; analyze_region_swap is no longer described as mutagenesis,
    and score_ism is listed. README, docs/IMPLEMENTATION_GUIDE.md and docs/MCP_WALKTHROUGH.md
    stated 24 tools, 22 tools and 6 oracles; there are 27 and 8.
  • tests/test_mcp_agent_facing_text.py reads the text the way a client receives it, through
    mcp.list_tools() and the prompt functions. Every "N tracks" claim about AlphaGenome is
    checked against the metadata, which needs no backgrounds, so CI runs it too; every "N tracks"
    claim about the other oracles is checked against the shipped backgrounds. It also checks that
    the example searches and each truncation-note example return tracks, that assay types match
    the metadata, and pins each behaviour fix.

Fixed — a UCSC hiccup turned CI red

  • The browser smoke job failed on an outage shape its guard did not know. Committed reports
    fetch hg38 from UCSC when opened. On 2026-11-15 UCSC answered without CORS headers, Chromium
    reported blocked by CORS policy and a bare net::ERR_FAILED, and the push build of #249
    failed. That change touched only Python imports, and the same corpus passed seven minutes
    later. tests/browser_harness.py now recognises both signatures. The rule that every failing
    URL must belong to the reference host still holds, so a CORS failure anywhere else still fails.
    The run's errors are pinned verbatim in tests/test_browser_outage_guard.py.

Fixed — found by the v0.8.0 release audit

The pre-release run of audits/AUDIT_CHECKLIST.md is recorded in
audits/2026-11-16_v080_release_audit.md.

  • A committed walkthrough carried percentiles the pinned null does not give.
    variant_analysis/SORT1_chrombpnet was regenerated on 2026-08-28 against a ChromBPNet null that
    is not the published one. It reported effect "≥99th (1.18× null max)" and activity 0.885, while
    the pinned chrombpnet_pertrack.npz (sha256 d4d6488c…) gives 0.9995 and 0.906, the values
    v0.7.5 shipped. It has been regenerated. A new test,
    tests/test_committed_percentiles_match_the_pinned_null.py, recomputes every committed
    percentile in examples/**/example_output.json (2,412 values across 18 outputs) from the
    pinned backgrounds and fails on any mismatch. It skips where backgrounds are absent, as in CI.

  • single_oracle_quickstart.ipynb no longer downloads ~70 GB on "Run All". Its conservation
    cell called build_variant_report(..., show_conservation=True) unconditionally, so the
    notebook recommended to first-time users fetched seven genome-wide bigwigs (66 GiB) on a fresh
    data dir. The cell is now behind RUN_CONSERVATION = False. With the flag off, it prints each
    missing track with its size and the chorus conservation download command. The notebooks
    README states the disk requirement, and tests/test_quickstart_conservation_cell_is_gated.py
    pins the gate.

  • GPN-Star, phyloP 100-way and phastCons 100-way are credited. #246 began downloading and
    drawing them, but docs/THIRD_PARTY.md named none of them. It now has an Annotation and
    conservation data
    section, linked from the README Acknowledgments:

    • GPN-Star (Ye et al., Nature 2026; songlab/gpn-star-scores at the pinned revision,
      Apache-2.0);
    • phyloP and phastCons (Pollard et al. 2010; Siepel et al. 2005; UCSC multiz100way, under
      UCSC's terms);
    • GENCODE v48.

    The igv.js entry described the 3.1.1 bundle #246 replaced. It now names 3.8.4 with its sha256,
    quotes igv.js's MIT notice, and lists the libraries bundled inside.

  • An LDlink failure wrote the LDlink token to the MCP server's log. fetch_ld_variants raised
    its redacted error from exc, which kept the original requests exception chained; that
    exception's message is the full request URL, token included. _safe_tool logged the whole
    chain, and with CHORUS_MCP_DEBUG=1 returned it too. The fixes:

    • both raises are now from None;
    • _safe_tool scrubs token values and token= parameters before it logs or returns anything;
    • urllib3, which logs request URLs at DEBUG, is held at INFO for the call.

    tests/test_credentials_never_reach_an_error_message.py drives an outage through an MCP tool.

  • rsID input to analyze_variant_multilayer, discover_variant and
    discover_variant_cell_types never worked.
    _resolve_rsid_to_position read v.pos, but
    LDVariant names the field position, so every successful lookup raised AttributeError.
    This has been the case since the feature arrived (#84). Only the failure paths had ever run.
    tests/test_rsid_position_resolves.py covers both branches.

  • The login advice named a command that no longer runs. Every shipped env resolves
    huggingface_hub 1.26, where huggingface-cli exits 1. Error messages, setup and health output,
    and the README now say hf auth login. tests/test_error_recovery.py fails if huggingface-cli
    returns to a string literal in the package.

  • Causal reports described the effect null as "a background of common variants". It is about
    18,000 variants sampled from the regulatory regions each assay measures. The footnote, the
    README and docs/NORMALIZATION_GUIDE.md now say so, and
    tests/test_reports_describe_the_real_reference_class.py forbids the old phrase.

  • Three test-harness gates no longer depend on the host or on UCSC.

    • The browser meta-test consults the outage guard when the channel it expects stays silent, so
      a UCSC outage skips rather than fails. That outage is what turned CI red on 4258b6f. A caught
      mutation, uncaught JS, or a failing non-UCSC host still never skips.
    • tests/test_data_dir_resolution.py runs against a throwaway install root. It passes on
      installs that use a data dir, and no longer rewrites the real chorus_data_dir.txt, which it
      used to do for about 4 s.
    • Integration tests and scripts/gate_end_to_end_determinism.py find hg38 through the data dir
      (tests/reference_genome.py). They used to look in the repo, which skipped all six oracle
      smoke tests on a host set up the documented way.
  • scripts/regenerate_examples.py looked for hg38 in the repo and exited 0 when every example
    failed.
    It now resolves the genome through the data dir and exits 1, naming the examples that
    failed.

  • The MCP end-to-end release test dropped CUDA_VISIBLE_DEVICES from the server's environment.
    The test passes chorus-mcp an explicit environment whitelist, and that whitelist left out the
    GPU mask. So on a shared host the server could initialise every GPU, including ones the run was
    not given. The whitelist now also forwards CUDA_VISIBLE_DEVICES, CHORUS_DATA_DIR and
    CHORUS_MCP_OUTPUT_DIR.

  • A browser cache from another playwright release read as 47 report failures. The browser
    checks skipped when no Chromium was cached at all. A cached Chromium of the wrong build passed
    that check and then failed every launch with "Executable doesn't exist". That is now a skip that
    names the build and the playwright install chromium fix; any other launch error still fails.

Changed

  • One hf_download_flat helper replaces two line-for-line copies of the HF
    download-and-flatten block. They differed only in whether they passed revision=, which is
    precisely how the unpinned download shipped.
  • The new surface is documented, which it was not: README gains a Conservation tracks
    section and its MCP tool count goes 24 → 27 with the three new tools listed;
    docs/API_DOCUMENTATION.md gains Annotations & Conservation. Three copies of the
    show_conservation blurb described the logo track as IGV's dynseq showing
    clip(1 - entropy, 0, 1) — it is chorus's own stacked-logo track showing p(base) × (2 - H)
    on a 0–2 bit scale — and stated ~25 GB of downloads where the logo track needs ~70 GB.
  • chorus/analysis/static/igv.min.js was replaced (1.35 MB → 1.50 MB, dropping the bundled
    jQuery 3.3.1). Recorded because it was an undeclared vendored-dependency bump inside a
    feature change. The 19 committed example reports still inlined the old bundle, and the
    browser check renders committed reports — so the new bundle shipped with no CI coverage.
    Partly closed: rs12740374_SORT1_chrombpnet_report.html, one of the two reports
    CHORUS_BROWSER_SMOKE=1 renders, is regenerated and now embeds the new bundle. Verified with
    Playwright/Chromium against test_committed_reports_render_in_a_browser.py — the exact CI
    smoke subset (11 passed, 1 skipped) and the full 19-report corpus (46 passed) — so the new
    bundle is confirmed to paint every canvas with no console or page errors, not merely inferred
    from size/content. The other smoke report (rs12740374_SORT1_cherimoya_report.html) still
    carries the old bundle — regenerating it needs the chorus-cherimoya environment, unavailable
    in the session that did this — and so do the other 17 committed reports outside the CI smoke
    subset. Still open, though now a coverage-completeness gap rather than an unverified change.
  • Conservation tracks and the annotation catalog get a worked example. Neither had one:
    show_conservation, chorus conservation, chorus annotation and AnnotationStore appeared
    in no notebook. examples/notebooks/single_oracle_quickstart.ipynb gains a Conservation
    Tracks
    section reusing the notebook's own GATA1 variant — build_variant_report(..., show_conservation=True) plus a short AnnotationStore.list_annotations() listing — executed
    with real data (all three conservation sources downloaded, ~24 GB combined) rather than left
    unexecuted; the notebook prints the resulting 103.5 MB report's path and size but writes it to
    a gitignored scratch directory rather than committing it. Still worth a dedicated walkthrough
    if the CLI/MCP side of the annotation catalog needs one too.
  • A dedicated conservation/ walkthrough was added, then pulled before release.
    examples/walkthroughs/conservation/SORT1_rs12740374 scored rs12740374 with ChromBPNet
    DNASE:HepG2 next to the three conservation scores at the variant's base. The v0.8.0 release
    audit found its committed percentiles (effect "≥99th (1.18× null max)", activity 0.885) are not
    reproduced by the pinned ChromBPNet null, which gives 0.9995, inside the null's support (max
    1.606), and 0.906. Its "saturating effect" headline therefore did not hold. It returns once
    regenerated against the pinned background; restore it from eab109c. The fix that came with
    it stays: README.md and docs/API_DOCUMENTATION.md documented chorus conservation list and
    chorus conservation download gpn_star, but the CLI has status (not list), and download
    requires --track.

Known limitations

These are pre-existing issues the release audit rated "fix before release" (P1) but left for a
later release; they are not regressions. The audit report has the evidence for each.

  • Variants near a chromosome's 3' end cannot be scored. predict_variant_effect widens the
    region to the model's input window without clamping it at the chromosome end. So any variant
    within half a window of a q-terminus raises InvalidRegionError. Examples: MC1R rs1805007 with
    AlphaGenome, and every chrM variant with Enformer, Borzoi and AlphaGenome.
  • LegNet mis-scores insertions. For example, G>GA scores −0.41 with percentile −0.985, where
    about +0.02 is expected. Score SNVs only with LegNet until this is fixed.
  • BATv1 (#248) is not in this release. It is
    under review: its nulls are not yet mirrored to the pinned dataset, and three issues block the
    merge.
  • The conservation walkthrough is pulled until it is regenerated against the pinned
    background (see Changed).
  • Both AlphaGenome backends stay on alphagenome 0.7.0. #249 makes the imports work under
    alphagenome 2.0, but the pins hold the versions the shipped backgrounds were built and
    validated with. Moving them waits on a fresh env build and a GPU check that predictions still
    match the backgrounds.
  • No env was built fresh from the pinned env files for this release. The pins reproduce the
    installed envs the backgrounds and validations ran in; the build itself was not repeated.
  • Shipped examples and notebooks:
    • Four walkthrough READMEs quote result tables that disagree with their own
      example_output.md.
    • cherimoya_quickstart.ipynb hardcodes ../../genomes/hg38.fa and describes the 5-fold
      ensemble, although fold 0 has been the default since 0.7.2.
    • comprehensive_oracle_showcase.ipynb prints AlphaGenome's 5,731 metadata rows as its track
      count, and states one insight ("H3K27ac +80") that matches no computed number.
    • The two committed causal reports keep the old null footnote until they are regenerated.
    • 18 of the 19 committed IGV reports still inline the previous igv.js (3.1.1). They render, and
      the browser check covers them.
  • Counts and paths:
    • Stale counts remain in some notebooks, in scripts/README.md, and in code comments. The
      README and the docs guarded by tests/test_documented_track_counts.py are clean, and the
      text the MCP server shows agents was corrected in #251.
    • The background builders and probes under scripts/ still expect <repo>/genomes/hg38.fa.
      Pass --reference, or symlink the data dir's genome there.
  • Enformer accepts a description such as "DNASE:K562" at validation that its env predict
    template then rejects.
    Pass the identifier (ENCFF…, CNhs…) from list_tracks; the MCP
    examples now do.
  • AnnotationStore reports GENCODE as "not downloaded" and lists its .bgz/.tbi index
    files as annotations.
    This affects list_annotations/describe_annotation too.
  • Loading AlphaGenome metadata in its env rewrites the tracked alphagenome_tracks.json in
    the package directory.