Fix #133: rerender_examples.py points at the real directory, and refuses to degrade - #161
Merged
Merged
Conversation
…ses to degrade This script has been dead since 2026-04-21. examples/applications/ became examples/walkthroughs/ in 340f30e and this was the only file in the repo still on the old name, so every invocation died with FileNotFoundError before doing any work. Anything added to it since was inert. The path fix alone makes things worse, which is why it was left broken. VariantReport.from_dict does not carry the per-bin prediction arrays the IGV panel is drawn from, so a rehydrated report is structurally valid, renders without complaint, and has no signal tracks. So the path fix ships only together with a guard that compares the candidate against the artefact it is about to replace. Measured on the current tree, all 14 rehydratable reports come back at 0.3-1.0% of their incumbent size: rs12740374_SORT1_multioracle_report.html 4.30 MB -> 0.01 MB (0.3%) chr5_1295046_T_G_TERT_alphagenome_report 8.45 MB -> 0.04 MB (0.5%) rs12740374_SORT1_enformer_report.html 6.70 MB -> 0.04 MB (0.5%) so every one is refused and the run exits 1 having written nothing. The 0.5 threshold sits nowhere near either regime: observed degradation is 100-900x, while a genuine renderer change moves size by a few percent. Two details that matter more than the threshold: * The guard renders to a temp file and only then replaces the target. Writing first and checking after would mean the artefact is already destroyed by the time the check runs -- and since the degraded output is valid, nothing downstream would detect it. * The multi-oracle consolidator is guarded too. It is the worst observed case (900x), and its markdown and JSON are only rewritten if the HTML survives, so a refused run leaves the directory internally consistent rather than half refreshed. --check reports without writing; --force overrides, and its help text says plainly that this is how 15 reports were silently degraded. Also corrects the module docstring, which carefully listed what was "not covered (require re-running the oracle)" while omitting the actual limitation: the IGV panel is lost EVERYWHERE, not just for causal reports. And enforces a rule that already existed. audits/AUDIT_CHECKLIST.md item 207 is a P0 checkbox stating the old path must only appear in audits/ historical snapshots -- a manual grep nobody ran, which this script was in violation of. It is now a test. resume.md's #133 row is updated from "do not just fix the path" to what was done. 899 fast tests pass; the end-to-end guard test passes under -m integration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #133.
scripts/rerender_examples.pyrefreshes every shipped example HTML from its saved JSON with no GPU and no model downloads. It has been dead since 2026-04-21, and the obvious one-line fix makes things worse — which is exactly why the issue said not to ship it alone.Half one: it pointed at a directory that does not exist
examples/applications/becameexamples/walkthroughs/in 340f30e. This was the only file in the repo still on the old name, so every invocation since died withFileNotFoundErrorbefore doing any work. Anything added to it in the meantime was inert.Half two: fixing the path alone silently destroys 15 artefacts
VariantReport.from_dictdoes not carry the per-bin prediction arrays the IGV panel is drawn from. A rehydrated report is structurally valid, renders without complaint, and has no signal tracks. No exception, no warning, and a diff that reads as a successful refresh.So the path fix ships only together with a guard that compares the candidate output against the artefact it is about to replace. Measured on the current tree, all 14 rehydratable reports come back at 0.3–1.0% of their incumbent size:
rs12740374_SORT1_multioracle_report.htmlchr5_1295046_T_G_TERT_alphagenome_report.htmlrs12740374_SORT1_enformer_report.htmlrs12740374_SORT1_CEBP_validation_report.htmlEvery one is refused, and the run exits 1 having written nothing. The 0.5 threshold sits nowhere near either regime: observed degradation is 100–900×, while a genuine renderer change moves size by a few percent.
Two details that matter more than the threshold
--checkreports without writing.--forceoverrides, and its help text says plainly that this is how 15 reports were silently degraded.Also
audits/AUDIT_CHECKLIST.mditem 207 is a P0 checkbox stating the old path must only appear inaudits/historical snapshots — a manual grep nobody ran, which this very script violated. Now a test. Dated audit files stay exempt; they record what was true when written.resume.md's rerender_examples.py is dead code, and fixing only its path silently destroys 15 shipped reports #133 row from "do not just fix the path" to what was actually done.Verification
-m integration: it runs the script for real, then asserts no file mtime changed, no file was created, no.rerender-tmpwas left behind,REFUSEDappears in the output, and the exit code is 1. A zero exit would let CI read a fully-refused run as a successful refresh.🤖 Generated with Claude Code