Skip to content

Fix #133: rerender_examples.py points at the real directory, and refuses to degrade - #161

Merged
lucapinello merged 1 commit into
mainfrom
fix/2026-08-04-rerender-refuse-degrade
Aug 4, 2026
Merged

Fix #133: rerender_examples.py points at the real directory, and refuses to degrade#161
lucapinello merged 1 commit into
mainfrom
fix/2026-08-04-rerender-refuse-degrade

Conversation

@lucapinello

Copy link
Copy Markdown
Contributor

Closes #133.

scripts/rerender_examples.py refreshes every shipped example HTML from its saved JSON with no GPU and no model downloads. It has been dead since 2026-04-21, and the obvious one-line fix makes things worse — which is exactly why the issue said not to ship it alone.

Half one: it pointed at a directory that does not exist

examples/applications/ became examples/walkthroughs/ in 340f30e. This was the only file in the repo still on the old name, so every invocation since died with FileNotFoundError before doing any work. Anything added to it in the meantime was inert.

Half two: fixing the path alone silently destroys 15 artefacts

VariantReport.from_dict does not carry the per-bin prediction arrays the IGV panel is drawn from. A rehydrated report is structurally valid, renders without complaint, and has no signal tracks. No exception, no warning, and a diff that reads as a successful refresh.

So the path fix ships only together with a guard that compares the candidate output against the artefact it is about to replace. Measured on the current tree, all 14 rehydratable reports come back at 0.3–1.0% of their incumbent size:

report incumbent rehydrated
rs12740374_SORT1_multioracle_report.html 4.30 MB 0.01 MB 0.3%
chr5_1295046_T_G_TERT_alphagenome_report.html 8.45 MB 0.04 MB 0.5%
rs12740374_SORT1_enformer_report.html 6.70 MB 0.04 MB 0.5%
rs12740374_SORT1_CEBP_validation_report.html 6.19 MB 0.05 MB 0.8%

Every one is refused, and the run exits 1 having written nothing. The 0.5 threshold sits nowhere near either regime: observed degradation is 100–900×, while a genuine renderer change moves size by a few percent.

Two details that matter more than the threshold

  • The guard renders to a temp file, then replaces. Writing first and checking after would mean the artefact is already destroyed by the time the check runs — and since the degraded output is valid, nothing downstream would detect it.
  • The multi-oracle consolidator is guarded too. It is the worst observed case (900×), and its markdown and JSON are only rewritten if the HTML survives the check, so a refused run leaves the directory internally consistent rather than half refreshed.

--check reports without writing. --force overrides, and its help text says plainly that this is how 15 reports were silently degraded.

Also

  • Corrects the module docstring. It carefully listed what was "not covered (require re-running the oracle)" while omitting the actual limitation: the IGV panel is lost everywhere, not just for causal reports.
  • Enforces a rule that already existed. audits/AUDIT_CHECKLIST.md item 207 is a P0 checkbox stating the old path must only appear in audits/ historical snapshots — a manual grep nobody ran, which this very script violated. Now a test. Dated audit files stay exempt; they record what was true when written.
  • Updates resume.md's rerender_examples.py is dead code, and fixing only its path silently destroys 15 shipped reports #133 row from "do not just fix the path" to what was actually done.

Verification

  • 899 fast tests pass, 13 skipped, 0 failures.
  • The end-to-end guard test passes under -m integration: it runs the script for real, then asserts no file mtime changed, no file was created, no .rerender-tmp was left behind, REFUSED appears in the output, and the exit code is 1. A zero exit would let CI read a fully-refused run as a successful refresh.
  • Working tree byte-identical after every run of the script.

🤖 Generated with Claude Code

…ses to degrade

This script has been dead since 2026-04-21. examples/applications/ became
examples/walkthroughs/ in 340f30e and this was the only file in the repo still on
the old name, so every invocation died with FileNotFoundError before doing any
work. Anything added to it since was inert.

The path fix alone makes things worse, which is why it was left broken.
VariantReport.from_dict does not carry the per-bin prediction arrays the IGV panel
is drawn from, so a rehydrated report is structurally valid, renders without
complaint, and has no signal tracks. So the path fix ships only together with a
guard that compares the candidate against the artefact it is about to replace.

Measured on the current tree, all 14 rehydratable reports come back at 0.3-1.0% of
their incumbent size:

  rs12740374_SORT1_multioracle_report.html   4.30 MB -> 0.01 MB  (0.3%)
  chr5_1295046_T_G_TERT_alphagenome_report   8.45 MB -> 0.04 MB  (0.5%)
  rs12740374_SORT1_enformer_report.html      6.70 MB -> 0.04 MB  (0.5%)

so every one is refused and the run exits 1 having written nothing. The 0.5
threshold sits nowhere near either regime: observed degradation is 100-900x, while a
genuine renderer change moves size by a few percent.

Two details that matter more than the threshold:

  * The guard renders to a temp file and only then replaces the target. Writing
    first and checking after would mean the artefact is already destroyed by the
    time the check runs -- and since the degraded output is valid, nothing
    downstream would detect it.
  * The multi-oracle consolidator is guarded too. It is the worst observed case
    (900x), and its markdown and JSON are only rewritten if the HTML survives, so a
    refused run leaves the directory internally consistent rather than half
    refreshed.

--check reports without writing; --force overrides, and its help text says plainly
that this is how 15 reports were silently degraded.

Also corrects the module docstring, which carefully listed what was "not covered
(require re-running the oracle)" while omitting the actual limitation: the IGV panel
is lost EVERYWHERE, not just for causal reports.

And enforces a rule that already existed. audits/AUDIT_CHECKLIST.md item 207 is a
P0 checkbox stating the old path must only appear in audits/ historical snapshots --
a manual grep nobody ran, which this script was in violation of. It is now a test.
resume.md's #133 row is updated from "do not just fix the path" to what was done.

899 fast tests pass; the end-to-end guard test passes under -m integration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lucapinello
lucapinello merged commit 6410d36 into main Aug 4, 2026
1 check passed
@lucapinello
lucapinello deleted the fix/2026-08-04-rerender-refuse-degrade branch August 4, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rerender_examples.py is dead code, and fixing only its path silently destroys 15 shipped reports

1 participant