Skip to content

regen(examples): ChromBPNet expm1 CDFs + three latent staleness fixes - #117

Merged
lucapinello merged 1 commit into
mainfrom
fix/2026-07-31-phase6-chrombpnet-regen
Jul 31, 2026
Merged

regen(examples): ChromBPNet expm1 CDFs + three latent staleness fixes#117
lucapinello merged 1 commit into
mainfrom
fix/2026-07-31-phase6-chrombpnet-regen

Conversation

@lucapinello

Copy link
Copy Markdown
Contributor

Phase 6 regeneration after the ChromBPNet expm1 fix (#113) and the HF background rebuild. Regenerating also surfaced three artefacts that were stale against earlier merged changes and one blocking issue that predates this work. Everything is separated below so nothing gets misattributed to expm1.

1. The intended change — expm1

ChromBPNet raw counts come down by exactly the window softmax mass w; percentile rankings are unchanged:

ref alt log2FC effect %ile activity %ile
before 287.857 747.653 +1.37394 0.9995 0.9059
after 287.237 747.014 +1.37581 0.9995 0.9059
Δ −0.620 −0.639 +0.0019

klf1 locus profile: max 127.333 → 127.307, mean 0.462 → 0.461. Same signature.

2–4. Latent staleness this surfaced (not caused by expm1)

Notebooks

Re-executed on H100 with zero error outputs: klf1 14/14 cells, advanced_multi_oracle_analysis 57/57, comprehensive_oracle_showcase 38/38. The documented chorus kernelspec (examples/notebooks/README.md:45) had to be registered first — it is not registered by chorus setup. Re-execution also removed a foreign-home path leak (/Users/jieconglin) from klf1's outputs.

⚠️ Blocked, and it predates this PR — needs your decision

rs12740374_SORT1_legnet_report.html and rs12740374_SORT1_multioracle_report.html are left at their origin/main bytes on purpose.

#99 (2026-06-18) added "tile LegNet across the locus for the IGV browser", but the legnet artefact was last committed 2026-06-17 (#95, single-window). The tiling has therefore never been applied. Applying it embeds a locus-wide 1 bp array:

consolidation size IGV tracks
with legnet .pkl (i.e. #99 as intended) 137 MB / consolidated 145 MB 21
without legnet .pkl 9.55 MB 18 — legnet's 3 dropped
origin/main (committed) 9.93 MB 21

Both regenerated variants are wrong: the first exceeds GitHub's 100 MiB file limit (the push was rejected by the pre-receive hook), the second silently loses tracks. The committed 9.93 MB / 21-track artefact came from a pre-#99 single-window .pkl and is not reproducible with current code. So #99 has been unshippable since it merged.

Options, all needing a call from you: bin/downsample the embedded IGV arrays, revert the tiling to single-window, or move these HTMLs to Git LFS.

Consequence to be aware of while merging: the consolidated example_output.{md,json} in this PR carry the corrected ChromBPNet numbers, but the consolidated HTML does not, until the above is resolved.

Other notes

  • legnet and alphagenome were re-run only to regenerate their gitignored *_variant_report.pkl, without which --consolidate silently degrades to "JSON only (no IGV predictions)".
  • AlphaGenome is not deterministic on this box: two consecutive runs differ on all 64 raw values (e.g. ref_value 2573.0 → 2568.0), though quantile_score was stable in all 58 cases. Their small numeric deltas here are that noise. Worth knowing for AUDIT_CHECKLIST §13, which asserts identical predictions — that gate passed earlier only because AlphaGenome wasn't installed yet.

Fast suite: 477 passed, 4 skipped, 0 errors.

🤖 Generated with Claude Code

The ChromBPNet per-track CDFs changed (#113 + the HF background rebuild),
so every committed ChromBPNet artefact drifted. Regenerating them also
surfaced three artefacts stale against EARLIER merged changes that were
never re-executed. All four effects are separated so nothing is
misattributed to expm1.

1. expm1 (the intended change). ChromBPNet raw counts come down by exactly
   the window softmax mass w; percentile RANKINGS unchanged:
     SORT1 DNASE:HepG2  ref 287.857 -> 287.237 (-0.620)
                        alt 747.653 -> 747.014 (-0.639)
                        log2FC +1.37394 -> +1.37581
                        effect %ile 0.9995, activity %ile 0.9059 (unchanged)
     klf1 locus profile max 127.333 -> 127.307, mean 0.462 -> 0.461

2. SORT1_chrombpnet was stale since 2026-05-09, predating the 2026-06-17
   windowing fix (#94). That is the large jump (ref 48.53 -> 287.24, log2FC
   +0.318 -> +1.376), NOT expm1 — the post-#94 multioracle artefact already
   carried +1.37394, which the regenerated value matches to within the
   expm1 residual.

3. klf1's EPInformer-seq cells were stale against the 2026-06-04
   "widewin-roadmap retrain (sparse H3K27ac)" weights swap on
   lucapinello/chorus-epinformerseq-v2 (HF commit 2654c05b7c) — committed
   alongside it but never re-executed. Hence K562 H3K27ac mean 1.771 ->
   0.132. Verified NOT device-dependent: cpu vs cuda agree to 0.006 on a
   3.78 range (0.16%). epinformerseq.py:416 calls hf_hub_download with no
   `revision`, which is what let this drift silently.

4. The AlphaGenome report was stale against the CAGE minus-strand dedup
   (#99/#101); it still carried both strands. all_scores 61 -> 32 is
   entirely CAGE:HepG2 58 -> 29 — the duplicate #101 exists to remove. No
   track lost: same 4 unique descriptions before and after.

Notebooks re-executed on H100, zero error outputs: klf1 14/14 cells,
advanced_multi_oracle 57/57, comprehensive_showcase 38/38. Registering the
documented `chorus` kernelspec (examples/notebooks/README.md:45) was
required first. Re-execution also dropped a foreign-home path leak
(/Users/jieconglin) from klf1's outputs.

DELIBERATELY NOT REGENERATED — blocked, needs a decision:
rs12740374_SORT1_legnet_report.html and rs12740374_SORT1_multioracle_report.html
are left at their origin/main bytes. #99 (2026-06-18) added "tile LegNet
across the locus for the IGV browser", but the legnet artefact was last
committed 2026-06-17 (#95, single-window), so the tiling has never been
applied. Applying it embeds a locus-wide 1-bp array and the report becomes
137 MB / the consolidated one 145 MB — both above GitHub's 100 MiB file
limit, i.e. unpushable. Dropping legnet's .pkl instead yields a shippable
9.55 MB but silently loses legnet's 3 IGV tracks (21 -> 18). The committed
9.93 MB / 21-track artefact came from a pre-#99 single-window .pkl and is
not reproducible with current code. So #99 has been unshippable since it
merged. Options: bin/downsample the embedded IGV arrays, revert the
tiling, or move these HTMLs to Git LFS.

Consequence: the consolidated example_output.{md,json} here carry the
corrected ChromBPNet numbers while the consolidated HTML does not, until
that is resolved.

Fast suite: 477 passed, 4 skipped, 0 errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lucapinello
lucapinello merged commit 707badb into main Jul 31, 2026
1 check passed
@lucapinello
lucapinello deleted the fix/2026-07-31-phase6-chrombpnet-regen branch July 31, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant