Fix MemoryError during HTML report generation (port of iLEAPP #1746) - #397
Merged
Conversation
Report generation loaded each artifact's entire .temphtml page into memory and rebuilt it via string concatenation to inject the navigation sidebar. For large extractions an artifact page can reach several GB, so this peaked at roughly 3x the page size and raised MemoryError. Replace the whole-file read/concat with stream_insert_sidebar_code(), which copies the page to its final location in fixed-size chunks. The sidebar placeholder is near the top of every page, so only a small head buffer is retained while searching for it (handling a placeholder split across a chunk boundary); the large table body that follows is streamed with shutil.copyfileobj. Output is byte-identical to the previous approach. Ported from iLEAPP commit 0a4aee10 (issue #1746, authored by JSup). Co-Authored-By: JSup <hanjisang0914@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Ports the iLEAPP MemoryError fix (abrignoni/iLEAPP#1746, fixed by JSup in iLEAPP commit 0a4aee10) to RLEAPP. Report generation used to read each artifact's entire .temphtml page into memory and rebuild it with string concatenation to inject the navigation sidebar. On large returns an artifact page can reach several GB, so that peaked at roughly three times the page size and raised MemoryError.
The new stream_insert_sidebar_code() copies each page to its final location in fixed size chunks. The sidebar placeholder sits near the top of every page, so only a small head buffer is held while searching for it (including a placeholder split across a chunk boundary). The large table body that follows is streamed with shutil.copyfileobj. Output is byte identical to the old approach.
Verification
To be clear about the validation boundary: the streaming path itself is exercised by the synthetic e2e run and the local regression checks. No multi GB real return was run against this port.
🤖 Generated with Claude Code