Repository navigation
v0.7.4 — Sei's numbers change
Chorus v0.7.4
Sei's numbers change in this release. Its variant effects were missing a correction the upstream
authors specify, and its background covered 40 of the 21,947 tracks it predicts. If you have published
Sei results from 0.7.3 or earlier, they were computed without the nucleosome-occupancy adjustment and
against a 40-row null. Nothing else moves: no other oracle's predictions, percentiles or committed
examples are affected.
Fixed
-
Sei's nucleosome-occupancy correction was absent in two independent ways
(#224,
#225).SeiNormalizer.normalizecomputed
sum_altfrom the reference array, so both scaling factors were exactly 1.0 — a no-op. And it was
never called:sei.pybuilt the normalizer, required it non-None, and never invoked it. The
correction is described in its own docstring as "recommended by Sei authors" and is
sc_hnorm_varianteffectin FunctionLab/sei-framework;normalizeis now a faithful port, asserted
withnp.array_equalagainst a transcription rather than merely close. Measured on three real
variants, applying it moves every one of the 40 sequence classes, by 12–48% of the variant's
largest effect, and changes the top-ranked class for 2 of 3. At chr11:5247500 the correction is larger
than the signal (median |Δ| 0.0181 vs median |effect| 0.0170).The correction is defined over a ref/alt pair, which a per-sequence
_predictcannot express, so
OracleBasegained_predict_alleles— a seam whose default is exactly the per-allele loop it
replaced, leaving the other eight oracles untouched.SeiNormalizer.equalizegeneralises the pair to
n alleles and is bit-for-bit upstream for ref + 1 alt, so chorus's result schema stays identical
across oracles. -
21,907 of Sei's 21,947 tracks produced no
raw_score
(#231). They classify ashistone_marks/
tf_binding/chromatin_accessibility, whose configs carrywindow_bpof 2001/501/501 because
those layers were built for binned oracles — and windowing Sei's 1-element output returnsNone.
score_track_effectalready had a "Full output scoring (LentiMPRA, Sei, etc.)" branch that Sei's
profiles never reached. Scoring now routes on the data shape, so a track carrying one value takes that
branch whatever its layer. Provably scoped: the only tracks affected are scalar tracks in a windowed
layer, which is Sei's profiles alone. -
chorus/analysis/discovery.pyraisedAttributeErrorfor Sei and skipped two oracles entirely
(#227,
#229). It calledoracle.get_all_assay_ids(),
which Sei does not have (only a private_get_all_assay_ids), for an oracle listed in its own
multi-track set.alphagenome_ptandcherimoyawere in neither oracle set, so both returned
{"error": "Unsupported oracle"}despite being fully implemented. Cherimoya now iterates its 407
biosamples rather than all 1,518 experiments — measured 15.3 s per model, so 1.7 h instead of 6.5 h —
with both counts logged rather than silently capped. -
epinformerseq.list_cell_types()returned 1 of 11 cell types
(#227) — the instance's own cell rather than the
oracle's catalogue, while its background carries all 33 rows. LegNet fixed this exact bug and the fix
was never ported. -
MCP
list_tracksanswered for Sei from a stale literal and for ChromBPNet with models that do not
exist (#230,
#231). The Sei branch returned a hardcoded
["sequence-class"]without ever consulting the oracle — false for an oracle with 1,176 assay types.
ChromBPNet reported 172 cell types × 240 TFs, implying 41,280 available models against 744
that exist. Both derive fromdescribe_tracks()now, keeping their payload shapes; ChromBPNet states
num_chip_modelsand rules out the cross-product reading in words. LegNet and EPInformer-seq no
longer hold third copies of their constants. -
A third-party timeout could fail the browser job
(#228). igv.js reports a resource timeout in its
own words ("IGV error: Timed out") as well as Chromium'sERR_TIMED_OUT, and one unrecognised console
error disqualifies the whole outage skip — so a UCSC timeout reddened CI on a change that touched no
browser code.
Changed
-
Sei's background covers all 21,947 tracks, not 40
(#226). The 40-only scope was never a recorded
decision: the builder's docstring asserted "Sei outputs 40 regulatory classes" and everything
downstream inherited it. The network emits 21,907 chromatin profiles that a projection aggregates into
the 40,predict()accepts profile ids, and it returned real values whose percentile was always
None. Compute was already being spent —project()discarded the profiles — so the cost is storage:
2.8 MB → 1.5 GB. The builder importsSeiNormalizerrather than carrying a second copy of the
arithmetic, and writes ids in the oracle's ownTA#/CA#forms (verified to match
_target_assays_ids() | _class_assays_ids()exactly, 0 divergence, before the build ran).All 1,176 of Sei's assay-type strings previously classified as
other, which would have shipped
21,907 built, verified, unscoreable rows — §8's documented failure mode at 21,907× scale.
classify_track_layerroutes profiles through the existingclassify_chip_layer, giving 10,310
tf_binding, 9,225histone_marks, 2,372chromatin_accessibilityand 40
regulatory_classification, zero unclassified.The effect null is histone-equalised and the activity null is not, because the correction is defined
over an allele pair and a baseline is one sequence. Recorded in
docs/BACKGROUND_NULL_PROTOCOL.md's decision log. -
The pinned backgrounds revision is now
backgrounds-2026-08-16-sei-full
(#231), matching the rebuilt artefact.
Added
-
OracleBase.describe_tracks()— one call every oracle answers
(#227). Previously this took four different calls:
get_all_assay_ids()on four oracles,list_tracks()on cherimoya, a private_get_all_assay_ids()
on Sei, and nothing at all on chrombpnet, legnet or epinformerseq — so a contributor adding a tenth
oracle had no single method to implement, and consumers grew per-oracle branches that drifted from the
oracles they described. ReturnsTrackRecordobjects (chorus/core/tracks.py) whosetrack_idis
exactly whatpredict()accepts, plus assay, cell type, description, ahas_backgroundflag and
per-oracle extras.Concrete rather than abstract, deliberately: seven test-double subclasses across five test files
implement exactly the previous abstract set, and a ninth abstract method breaks every one at
instantiation. All nine oracles implement it, each count matching what is published — enformer 5,313,
borzoi 7,611, alphagenome and alphagenome_pt 5,168, cherimoya 1,518, sei 21,947, chrombpnet 753,
legnet 3, epinformerseq 33 — and each works beforeload_pretrained_model(). -
Near-ties at the top-N cutoff are now reported
(#235).discover_variant_effectsreturns
near_ties_at_cutoff, populated whenever the rank N / N+1 boundary in a layer is closer than 1% —
naming the kept and dropped track, the relative gap, and that which one is reported is not stable
between runs. Also logged as a warning. Selection is unchanged; this only stops a coin-flip being
presented as a result. It exists because the measured rank-12↔13 gap intf_bindingwas 4.25%
against a 4.29% worst-case cross-process drift, which is the one way the accepted drift can still
change a conclusion while no number moves visibly.
Documentation
- Audit coverage reached 18 of 18 checklist sections
(#223): GPU detection, per-track CDFs,
reproducibility and scientific determinism were the four never previously exercised. Every §4 figure
matched its published value; §3's documented trap (a bare probe reports 0 GPUs for both TensorFlow
envs, because their CUDA libs are only on the runner'sLD_LIBRARY_PATH) caught the auditor as
designed.
Known limitations
-
Cross-process drift is accepted rather than eliminated.
TF_DETERMINISTIC_OPS=1was measured to make Enformer bit-exact (4/4 runs, against a control failing 2/2 at 3.4% and 1.4%), but adopting it would require rebuilding the Enformer and ChromBPNet nulls and changing published numbers for both. Median drift is 0.016% and the publishedquantile_scoremoves at most 0.69%, so the trade is not worth it. The residual risk is which track gets reported, not the numbers:discovery.pycuts hard attop_n_per_layerand the rank-12↔13 gap can be smaller than the drift. Near-tie reporting is the follow-up. Seedocs/BACKGROUND_NULL_PROTOCOL.md's decision log, andaudits/2026-08-14_f8_localisation.mdfor the measurements (#233) and the per-oracle coverage map (#234) — cherimoya, legnet and epinformerseq are bit-exact cross-process; chrombpnet and sei remain unmeasured because the gate cannot target them. -
The committed-example staleness check reports rather than fails, following that decision. It compares each
committed example against the last semantic change toscorers.py, so this release's scoring fix
fires it. A regen sweep is currently the wrong response: committed examples cover alphagenome,
chrombpnet and enformer and no Sei, and the fix can only touch scalar tracks in a windowed layer,
so no committed number can move — while F8 (audits/2026-08-14_f8_localisation.md) remains open, so
regenerating would change Enformer's numbers for unrelated reasons. Integration-marked so it never
blocks CI. -
CI runs the fast and browser suites only. Three real defects in this release existed solely in the
integration suite, which needs a GPU and the per-oracle envs. Integration has to be run locally before
a release;audits/AUDIT_CHECKLIST.mdis the runbook.