regen(examples): ChromBPNet expm1 CDFs + three latent staleness fixes - #117
Merged
Merged
Conversation
The ChromBPNet per-track CDFs changed (#113 + the HF background rebuild), so every committed ChromBPNet artefact drifted. Regenerating them also surfaced three artefacts stale against EARLIER merged changes that were never re-executed. All four effects are separated so nothing is misattributed to expm1. 1. expm1 (the intended change). ChromBPNet raw counts come down by exactly the window softmax mass w; percentile RANKINGS unchanged: SORT1 DNASE:HepG2 ref 287.857 -> 287.237 (-0.620) alt 747.653 -> 747.014 (-0.639) log2FC +1.37394 -> +1.37581 effect %ile 0.9995, activity %ile 0.9059 (unchanged) klf1 locus profile max 127.333 -> 127.307, mean 0.462 -> 0.461 2. SORT1_chrombpnet was stale since 2026-05-09, predating the 2026-06-17 windowing fix (#94). That is the large jump (ref 48.53 -> 287.24, log2FC +0.318 -> +1.376), NOT expm1 — the post-#94 multioracle artefact already carried +1.37394, which the regenerated value matches to within the expm1 residual. 3. klf1's EPInformer-seq cells were stale against the 2026-06-04 "widewin-roadmap retrain (sparse H3K27ac)" weights swap on lucapinello/chorus-epinformerseq-v2 (HF commit 2654c05b7c) — committed alongside it but never re-executed. Hence K562 H3K27ac mean 1.771 -> 0.132. Verified NOT device-dependent: cpu vs cuda agree to 0.006 on a 3.78 range (0.16%). epinformerseq.py:416 calls hf_hub_download with no `revision`, which is what let this drift silently. 4. The AlphaGenome report was stale against the CAGE minus-strand dedup (#99/#101); it still carried both strands. all_scores 61 -> 32 is entirely CAGE:HepG2 58 -> 29 — the duplicate #101 exists to remove. No track lost: same 4 unique descriptions before and after. Notebooks re-executed on H100, zero error outputs: klf1 14/14 cells, advanced_multi_oracle 57/57, comprehensive_showcase 38/38. Registering the documented `chorus` kernelspec (examples/notebooks/README.md:45) was required first. Re-execution also dropped a foreign-home path leak (/Users/jieconglin) from klf1's outputs. DELIBERATELY NOT REGENERATED — blocked, needs a decision: rs12740374_SORT1_legnet_report.html and rs12740374_SORT1_multioracle_report.html are left at their origin/main bytes. #99 (2026-06-18) added "tile LegNet across the locus for the IGV browser", but the legnet artefact was last committed 2026-06-17 (#95, single-window), so the tiling has never been applied. Applying it embeds a locus-wide 1-bp array and the report becomes 137 MB / the consolidated one 145 MB — both above GitHub's 100 MiB file limit, i.e. unpushable. Dropping legnet's .pkl instead yields a shippable 9.55 MB but silently loses legnet's 3 IGV tracks (21 -> 18). The committed 9.93 MB / 21-track artefact came from a pre-#99 single-window .pkl and is not reproducible with current code. So #99 has been unshippable since it merged. Options: bin/downsample the embedded IGV arrays, revert the tiling, or move these HTMLs to Git LFS. Consequence: the consolidated example_output.{md,json} here carry the corrected ChromBPNet numbers while the consolidated HTML does not, until that is resolved. Fast suite: 477 passed, 4 skipped, 0 errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 6 regeneration after the ChromBPNet
expm1fix (#113) and the HF background rebuild. Regenerating also surfaced three artefacts that were stale against earlier merged changes and one blocking issue that predates this work. Everything is separated below so nothing gets misattributed toexpm1.1. The intended change —
expm1ChromBPNet raw counts come down by exactly the window softmax mass
w; percentile rankings are unchanged:klf1 locus profile: max 127.333 → 127.307, mean 0.462 → 0.461. Same signature.
2–4. Latent staleness this surfaced (not caused by
expm1)expm1residual, which is how I attributed it.2654c05b7c(2026-06-04) replacedper_cell_widewin/main.ptwith the "widewin-roadmap retrain (sparse H3K27ac)"; the notebook was committed alongside it but never re-executed. Hence K562 H3K27ac mean 1.771 → 0.132. Verified not device-dependent: cpu vs cuda agree to 0.006 absolute on a 3.78 range (0.16%). The mechanism that let it drift silently isepinformerseq.py:416—hf_hub_downloadwith norevision.all_scores61 → 32 is entirelyCAGE:HepG258 → 29 — precisely the duplicate regen(multioracle): keep the CAGE minus strand (the article's +1.52) #101 exists to remove. No track lost: same 4 unique descriptions before and after.Notebooks
Re-executed on H100 with zero error outputs: klf1 14/14 cells,
advanced_multi_oracle_analysis57/57,comprehensive_oracle_showcase38/38. The documentedchoruskernelspec (examples/notebooks/README.md:45) had to be registered first — it is not registered bychorus setup. Re-execution also removed a foreign-home path leak (/Users/jieconglin) from klf1's outputs.rs12740374_SORT1_legnet_report.htmlandrs12740374_SORT1_multioracle_report.htmlare left at their origin/main bytes on purpose.#99 (2026-06-18) added "tile LegNet across the locus for the IGV browser", but the legnet artefact was last committed 2026-06-17 (#95, single-window). The tiling has therefore never been applied. Applying it embeds a locus-wide 1 bp array:
.pkl(i.e. #99 as intended).pklBoth regenerated variants are wrong: the first exceeds GitHub's 100 MiB file limit (the push was rejected by the pre-receive hook), the second silently loses tracks. The committed 9.93 MB / 21-track artefact came from a pre-#99 single-window
.pkland is not reproducible with current code. So #99 has been unshippable since it merged.Options, all needing a call from you: bin/downsample the embedded IGV arrays, revert the tiling to single-window, or move these HTMLs to Git LFS.
Consequence to be aware of while merging: the consolidated
example_output.{md,json}in this PR carry the corrected ChromBPNet numbers, but the consolidated HTML does not, until the above is resolved.Other notes
*_variant_report.pkl, without which--consolidatesilently degrades to "JSON only (no IGV predictions)".ref_value2573.0 → 2568.0), thoughquantile_scorewas stable in all 58 cases. Their small numeric deltas here are that noise. Worth knowing forAUDIT_CHECKLIST§13, which asserts identical predictions — that gate passed earlier only because AlphaGenome wasn't installed yet.Fast suite: 477 passed, 4 skipped, 0 errors.
🤖 Generated with Claude Code