Fix excessive memory usage when saving EXR file with many channels #3176
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Scratch memory allocation in
ImageOutput::to_native_rectangle
was accidentally using the number of channels squared.We encountered a case where saving a 2K EXR file with 107 channels would use more than 50GB.
I don't have a simple isolated repro case. It requires writing tiles with a stride that does not match the stride in the image file. But hopefully the error in the code is clear.
Tests
I did not add a test, it's not clear there is a good mechanism to verify memory usage like this with a test.
Checklist:
have previously submitted a Contributor License Agreement
(individual, and if there is any way my
employers might think my programming belongs to them, then also
corporate).
(adding new test cases if necessary).