Skip to content

Fix MemoryError during HTML report generation (port of iLEAPP #1746) - #397

Merged
abrignoni merged 1 commit into
mainfrom
fix/report-sidebar-memoryerror
Aug 7, 2026
Merged

Fix MemoryError during HTML report generation (port of iLEAPP #1746)#397
abrignoni merged 1 commit into
mainfrom
fix/report-sidebar-memoryerror

Conversation

@abrignoni

Copy link
Copy Markdown
Owner

What this does

Ports the iLEAPP MemoryError fix (abrignoni/iLEAPP#1746, fixed by JSup in iLEAPP commit 0a4aee10) to RLEAPP. Report generation used to read each artifact's entire .temphtml page into memory and rebuild it with string concatenation to inject the navigation sidebar. On large returns an artifact page can reach several GB, so that peaked at roughly three times the page size and raised MemoryError.

The new stream_insert_sidebar_code() copies each page to its final location in fixed size chunks. The sidebar placeholder sits near the top of every page, so only a small head buffer is held while searching for it (including a placeholder split across a chunk boundary). The large table body that follows is streamed with shutil.copyfileobj. Output is byte identical to the old approach.

Verification

  • Ran the iLEAPP regression checks locally against this port: byte identical output across page shapes (placeholder at start, end, split across a chunk boundary, missing, empty file), and peak memory bounded under half the file size on a 16 MB+ synthetic page. Not committed as a test file because iLEAPP removed its copy from main after the fix merged; the port matches iLEAPP main as it stands.
  • pylint --disable=C,R on scripts/report.py: 10.00/10, zero warnings. admin/scripts/lint_changed.py against the merge base: PASS, no new warnings.
  • End to end: two artifact profile run (Discord Returns friendships and server metadata) against a fully synthetic return input, since no real returns corpus is on hand. Report generated clean: sidebar navigation present in every page, zero leftover placeholders, zero .temphtml remnants, no parser errors.

To be clear about the validation boundary: the streaming path itself is exercised by the synthetic e2e run and the local regression checks. No multi GB real return was run against this port.

🤖 Generated with Claude Code

Report generation loaded each artifact's entire .temphtml page into memory
and rebuilt it via string concatenation to inject the navigation sidebar.
For large extractions an artifact page can reach several GB, so this peaked
at roughly 3x the page size and raised MemoryError.

Replace the whole-file read/concat with stream_insert_sidebar_code(), which
copies the page to its final location in fixed-size chunks. The sidebar
placeholder is near the top of every page, so only a small head buffer is
retained while searching for it (handling a placeholder split across a chunk
boundary); the large table body that follows is streamed with
shutil.copyfileobj. Output is byte-identical to the previous approach.

Ported from iLEAPP commit 0a4aee10 (issue #1746, authored by JSup).

Co-Authored-By: JSup <hanjisang0914@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@abrignoni abrignoni added the Needs Cross Core Leveling for a PR that needs to be applied across other cores. remove once leveled to all. label Aug 7, 2026
@abrignoni
abrignoni merged commit ee8d70e into main Aug 7, 2026
4 checks passed
@abrignoni
abrignoni deleted the fix/report-sidebar-memoryerror branch August 7, 2026 05:37
@abrignoni abrignoni removed the Needs Cross Core Leveling for a PR that needs to be applied across other cores. remove once leveled to all. label Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant