Skip to content

v0.7.2

Choose a tag to compare

@lucapinello lucapinello released this 12 Aug 01:48
· 80 commits to main since this release

Changed

  • The display ceiling is 4.0 instead of 3.0, which reveals peaks that were being flattened. Nothing is rescaled: the band is still (v − floor) / (peak − floor), so 1.0 still means the track's genome-wide p99 and every value below 3.0 is the number it always was. Only the clip point moves. Measured share of each track's clipped mass that this recovers: ChromBPNet DNase 43%, Cherimoya DNase 41%, AlphaGenome H3K27ac 33%, CAGE 25%, AlphaGenome DNase 24%. On the SORT1 panel the fraction of bins sitting at the ceiling roughly halves — H3K27ac 0.0046 → 0.0010, ChromBPNet 0.0030 → 0.0018, CEBPA 0.0066 → 0.0042, Cherimoya 0.0103 → 0.0080, CAGE 0.0126 → 0.0097.

    The cost is uniform: a value of 3.0 now occupies 75% of the axis rather than all of it, so every track reads correspondingly shorter. Uniform is the point — raising the ceiling for everything preserves cross-oracle comparability exactly, where a per-track ceiling would destroy the one property that makes stacked panels readable together.

    Going higher does not pay. To clip nothing a track needs its genome-wide maximum on the axis, which is ~196 for ChromBPNet DNase and ~160 for AlphaGenome DNase in band units; at that ceiling a p99-level peak occupies 0.5% of the height and every panel reads flat.

    One consequence worth knowing: the ceiling clips before pooling, so mean-pooled tracks move too. AlphaGenome DNase's displayed peak went 2.92 → 3.81 despite never touching the old ceiling, because its native values were being clipped at 3.0 before being averaged. The new value is the mean of unclipped data, which is more faithful, but it is a change rather than a pure display tweak.

  • igv.js now reduces every track to pixels with max, not a two-name oracle list. A report emits ~349 bp display features, and igv.js reduces those to pixels whenever there are more features than pixels — about 3:1 on a 1 Mb panel. That second reduction was "max" if source_model in ["chrombpnet", "legnet"] else "mean", so Cherimoya and AlphaGenome's 1 bp CAGE got mean-reduced in the browser immediately after the feature stage had measured them into max: the original 5.5× dilution, one stage further down.

    Measured on the SORT1 panel at the browser's 3:1 ratio, the peak height mean costs: LegNet 2.33×, AlphaGenome DNase 1.56×, ChromBPNet 1.38×, CAGE 1.31×, Cherimoya 1.14×, H3K27ac 1.00×. Two things make that unacceptable rather than merely lossy — it costs unequally, and 1.38× for ChromBPNet against 1.14× for Cherimoya is a 1.2× relative distortion between the two tracks a cross-oracle panel exists to compare; and mean cancels signed tracks against themselves, which is why LegNet is the worst case.

    max loses no peak anywhere. Its cost is one small floor lift (Cherimoya 0.000 → 0.074 of a 0–3 axis) and roughly doubled saturation, still well under the 0.075 readability limit (CAGE 0.013 → 0.032 is the highest).

    One exception: log-scaled tracks keep mean. The log band compresses the top of the range, so many more bins sit just under the ceiling and max promotes them over it — AlphaGenome CAGE's saturation went 0.003 → 0.023 (7.7×) and the clipped flat tops read as RNA-seq gene-body coverage rather than TSS spikes. Ink barely moved (0.186 → 0.192), so it is the ceiling and not the density doing the damage. mean costs that track 1.31× of peak height, the cheaper of the two harms.

    Note this takes the opposite default from the feature stage, and deliberately: that stage collapses ~349 native bins, where max lifted AlphaGenome DNase's floor to 0.707 and so has to be decided per track; the browser collapses 2–3 already-pooled features, where max has almost no opportunity to promote background. Mirroring the feature stage's per-track choice was considered and is also wrong here — it would send AlphaGenome DNase and the ChIP tracks to mean, costing 1.1–1.6× of peak for floor protection a 3:1 collapse does not need.

  • ⚠️ Cherimoya now uses fold 0 by default instead of the 5-fold ensemble, so its scores move. Anyone comparing numbers against 0.7.1 will see Cherimoya percentiles and log2FCs differ. This is the default model changing, not a regression.

    Agreed with CATv1's author, @jmschrei: five models complicate and slow most analyses, so fold 0 is the right default for an interactive tool and the ensemble is there when you want to dig deeper. Fold 0 also matches ChromBPNet's default fold, and the two nulls are built on the same reference sets — the Cherimoya null reproduces ChromBPNet's effect_counts=18672 and summary_counts=34004 exactly — so a percentile from one means what a percentile from the other means. Raw magnitudes are not comparable; see the bias note below.

    The 5-fold mean stays available as fold="ensemble" and has its own null (cherimoya_ensemble_pertrack.npz); a percentile is a rank against a background, so each mode is ranked against a background built with the same model. Selection is automatic — every Cherimoya prediction records its fold — and a mismatch raises rather than returning a plausible wrong number.

    Folds 1–4 now raise: no null ships for them, and ranking one against another fold's null returns a plausible wrong number rather than an approximation.

  • Documented: Cherimoya and ChromBPNet are not comparable at raw magnitude, because one is bias-corrected and the other is not. chorus loads ChromBPNet as chrombpnet_nobias, the "TF Model" that predicts the bias-corrected accessibility profile — ChromBPNet trains a Tn5/DNase bias model first and regresses its effect out. CATv1 does no such correction: the shipped checkpoints carry n_control_tracks: 0, its training config has controls = None, and its ATAC recipe uses GC-matched negatives with no control file. Measured on the four shared DNase experiments, fold 0 both sides:

    CATv1 vs window sum peak peak/sum
    chrombpnet_nobias (what chorus loads) 1.32× 3.40× 2.19×
    chrombpnet (bias-aware) 1.14× 1.02× 0.80×

    So CATv1 tracks the bias-aware model almost exactly and differs from the bias-corrected one by ~3.4× on peak height. Profile shape agrees regardless (rank correlation 0.95, peaks 18 bp apart) — it is height and sharpness that separate. Percentiles are unaffected, since each oracle is ranked against its own null (0.9325 vs 0.9550 at the SORT1 locus), which is why the cross-oracle panel reads consistently. This asymmetry is pre-existing, not introduced here, and switching the panel to bias-aware ChromBPNet is deliberately left as separate work.

  • Cherimoya is 3.6× faster on the 5-fold ensemble, and 17× faster in-process — contributed by @jmschrei in #165. Triton benchmarks its autotune candidates the first time a kernel sees a shape, which is right for training but was being re-run in every subprocess to serve a single forward pass, because run_code_in_environment spawns a fresh process per call. Chorus now enables Triton's on-disk autotune cache before importing cherimoya, so the winner is reused across processes. Measured on one H200, DNASE:ENCSR149XIL, single 2,114 bp window:

    mode before after
    use_environment=True, 5-fold ensemble 89.5 s 25.0 s 3.6×
    in-process (use_environment=False) 13.6 s 0.79 s 17.3×

    Predictions are bit-identical, verified in both modes rather than assumed: the ensemble window sum is 954.37564392006766 and the peak 11.103147742142609 before and after, to the last digit. It reuses the config autotune already chose; an unseen shape still falls back to benchmarking. The cherimoya integration suite got 3–4× faster on its own as corroboration (test_variant_effect_runs_end_to_end 178 s → 50 s).

Added

  • The Cherimoya documentation gaps are filled — contributed by @jmschrei in #166, landed via #170. Cherimoya was missing from the chorus setup --oracle list and from the download_pertrack_backgrounds pre-download loop, so a reader following the README could not have set it up; it was also absent from docs/THIRD_PARTY.md attribution, the CLAUDE.md env list and examples/notebooks/README.md. Adds the macOS caveat (Cherimoya is CPU-only on Apple Silicon: no MPS path in the model, and its triton>=3.5.1 pin ships no macOS wheel) and corrects the background size to ~162 MB. Verified against the artefacts: 1,518 tracks = 369 ATAC + 1,149 DNASE, cherimoya_pertrack.npz 161.7 MB.
  • The README's Cherimoya timing table is re-measured, because #165 invalidated it — its "~12 s per fold per call" was the autotune benchmark that change removes. The headline row moves from ~60 s to ~25 s. The section now also states, with numbers, why pinning a single fold is not a cheaper approximation of the default: on one window the five fold peaks are 8.24 / 15.47 / 15.34 / 11.08 / 7.65 against an ensemble peak of 11.10, so they disagree by 2.02× among themselves and any one lands between 0.69× and 1.39× of the ensemble — and the shipped background CDFs were built against the ensemble, so a single fold would be ranked against the wrong null.

Fixed

  • Two IGV display decisions were hardcoded per oracle name, and both were wrong for at least one track. They are now measured per track from the data being drawn.

    Pooling. A display bin covers ~349 bp of a 1 Mb panel, so the renderer must reduce 349 native values to one — mean or max. That was chosen from a list of oracle names, which is how Cherimoya (a BPNet-family 1 bp model, absent from the list) rendered its peak at 0.547 instead of 3.000 — a 5.5× display-only dilution on the same 0–3 axis as ChromBPNet, in a report whose purpose is cross-oracle comparison. Five candidate predictors were measured and all get the sign wrong on at least one oracle: resolution <= 1, per-bin max/p99 from the artefact (22 for Cherimoya against 65 for AlphaGenome — backwards), the artefact's signal mass above p99 (0.122 for ChromBPNet against 0.243 for AlphaGenome — backwards), profile density, and density × collapse factor (AlphaGenome and Cherimoya both emit DNase at 1 bp and both collapse 349 bins, yet max lifts one floor to 0.707 and the other to 0.000 — and Cherimoya is the denser). So it is measured instead: max-pooling can never lose a peak and mean-pooling can never lift a floor, so the only question is whether max lifts this track's floor, which costs one reduce over an array already in memory. It decides per track, which matters because AlphaGenome needs opposite answers for its own 1 bp and 128 bp tracks.

    Scale. floor=p95, peak=p99, linear assumes signal decays smoothly out of the background. That holds for accessibility and fails completely for base-resolution TSS assays, where the genome is a huge mass of near-zero plus a tiny population of enormous peaks. AlphaGenome CAGE has p95 = 0.0050 and p99 = 0.0405 against a max of 852, so every real TSS from strength 1 to 3000 rendered at exactly 3.00 — no dynamic range among peaks at all. Measured on the shipped SORT1 panel, 13.1% of CAGE's display bins were pinned at the ceiling, against 0.0–1.3% for the panels that read well.

    A track is now re-rendered on a log band (log1p, anchored p99.5/p99.9) when the linear band is measured to clip more than 4% of the bins it will actually draw, and the log band is kept only if it both brings saturation down and leaves the strongest feature at or above 1.0 — that second condition is what an earlier attempt lacked, where p99.9/p99.99 anchors "fixed" saturation by dropping CAGE's peak to 1.24 of 3.0 and erasing the track.

    It is measured on the panel because no genome-wide CDF statistic can predict it. Four were tried across the 20,366 tracks that carry a CDF, split into "must get the log band" (1,296) and "was working, must not move" (19,070), and every one overlaps: max/p99.9 (must-log down to 172, must-stay-linear p95 20.5 and max 4212), p99.9/p99 (5.7 vs 15.6), p99/p95 (3.0 vs 10.0), and predicted clip fraction (0.0028 vs 0.0045). max/p99.9 at a threshold of 50 briefly looked clean at 41× separation — only because ChromBPNet's ChIP tracks had been left out of the protected set. It would have log-scaled 130 tracks outside the must-log set: 102 ChromBPNet ChIP, 10 Enformer and 8 Borzoi CAGE, 7 AlphaGenome TF-ChIP, 2 ChromBPNet DNase and 1 Cherimoya DNase — including AlphaGenome's own TF-ChIP tracks, which is the most awkward case, since AlphaGenome is the oracle the rule exists for. And on a 10,000-point grid int(0.9999 × n) is the last slot, so that statistic is a ratio to the single extreme order statistic the null protocol explicitly warns against.

    Saturation also has to be measured as drawn, not natively: pooling is what creates it. CAGE's native clip rate is 0.005–0.014, indistinguishable from the ChIP tracks at 0.001–0.008, because CAGE is 1 bp and collapses 349 native bins into each display bin while ChIP is 128 bp and collapses 2. The separation only exists after pooling — 0.131 against ≤0.013, a 10× gap — which is where the trigger is applied.

    Saturation — not ink — is what makes a panel unreadable, and the two concerns stay separate: pooling protects the floor, the scale protects the peaks. Cherimoya inks 41% of its display bins and reads correctly, because only 1.3% of them clip. An "ink fraction" criterion was tried for pooling and flipped Cherimoya and ChromBPNet to mean, re-creating the original defect.

    The 7.5% limit is calibrated on the corpus, not on one panel. An earlier 4% came from the geometric midpoint of a single panel's gap; measured across all 346 subtracks of the 19 committed IGV panels it cuts through the middle of the population — 45 subtracks (13%) exceed it, including seven Enformer CAGE tracks at 0.042–0.063 that render acceptably and whose peaks the log band would compress, silently invalidating committed panels that were never regenerated. The real gap has nothing in it: the 22 subtracks at or above 0.0899 are exactly the AlphaGenome CAGE/ATAC/DNase panels at the SORT1 locus, and the next one down is 0.0656. The limit sits inside that gap, and the calibration is conservative in the right direction because it was measured under the old pooling — the pooling fix lowers saturation for exactly these dense 1 bp tracks.

    Acceptance also requires the log band to clear the limit or at least halve the clipping, not merely reduce it: a bare "did it go down" test is satisfied by an epsilon, so a track going 0.550 → 0.500 would be re-rendered, relabelled and still ship with half the panel pinned. A log band whose anchors collapse — reachable from shipped data, chrombpnet CHIP:HEK293:ZNF24 has p99.5 = −7.4e-07 and p99.9 = −3.3e-10, which max(x, 0) maps to the same 0.0 — renders a two-level barcode that would pass both tests, since clipping guarantees the peak; it is now rejected outright.

    Signed tracks are excluded from both measured decisions. "Does max-pooling lift the floor" is meaningless for a track with no floor at zero, and the answer is actively harmful: max over a bin holding a strong repression and a weak activation returns the activation, so the repressive half of the panel disappears. Measured on borzoi ENCFF734OLC+ (signed, 32 bp, 11 native bins per display bin) the measured choice flips mean → max and takes displayed saturation 0.000 → 0.138. 2,253 shipped tracks are signed (borzoi 1,543, AlphaGenome 667, Sei 40, LegNet 3); they keep the static geometry-based choice.

    A re-rendered track is labelled (log scale), because its 1.0 means genome-wide p99.9 rather than p99. Two same-assay panels in one report can legitimately land on different bands — BCL11A's two CAGE:K562 tracks measured 0.053 and 0.036 saturation and only the first escalated. The 0–3 axis was always per-track (1.0 is that track's percentile, never a shared raw value), so mixing bands is not new; shipping it unlabelled would be. Follows LegNet's existing (per-track norm) precedent, and every IGV legend now carries the same caveat — it previously asserted p95/p99 anchors and "tracks comparable" for every track unconditionally.

    Deliberately not extended to the matplotlib/CoolBox figure paths, which share rescale_for_display: they mean-smooth rather than max-pool, so they cannot manufacture ceiling bins, and CAGE's native clip rate (0.005–0.014) is already under the limit — wiring the escalation in there would be a no-op that churns every committed figure.

    Verified on the shipped SORT1 multi-oracle panel: exactly two tracks change — AlphaGenome CAGE (saturation 0.131 → 0.013, ink 0.503 → 0.132, peak still 3.00) and AlphaGenome DNase (0.090 → 0.000, ink 0.982 → 0.423, peak 3.00 → 2.92, from the pooling fix). The other ten, including every ChromBPNet, Cherimoya, LegNet and AlphaGenome ChIP track, are bit-identical. CHIP:K562:ZBTB11 — the ChromBPNet track the discarded CDF statistic ranked as more bimodal than CAGE — measures 0.000 saturation as drawn and is left untouched. tests/test_display_scale_is_measured_not_declared.py pins the scope of both rules, including that all three duplicated render paths (_igv_report, multi_oracle_report, causal) go through the measured decision — patching one of the three is how a change to this logic came back reporting byte-identical output.

    Not changed, deliberately: LegNet. Its genome-wide maximum maps to 2.6 of a 3.0 axis, so its scale is correct and the SORT1 window simply contains no strong promoter. MPRA activity is a compact bounded score, not a heavy-tailed count; making it look dramatic would overstate the data.