Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix excessive memory usage when saving EXR file with many channels #3176

Merged

Conversation

brechtvl
Copy link
Contributor

Description

Scratch memory allocation in ImageOutput::to_native_rectangle was accidentally using the number of channels squared.

We encountered a case where saving a 2K EXR file with 107 channels would use more than 50GB.

I don't have a simple isolated repro case. It requires writing tiles with a stride that does not match the stride in the image file. But hopefully the error in the code is clear.

Tests

I did not add a test, it's not clear there is a good mechanism to verify memory usage like this with a test.

Checklist:

  • I have read the contribution guidelines.
  • If this is more extensive than a small change to existing code, I
    have previously submitted a Contributor License Agreement
    (individual, and if there is any way my
    employers might think my programming belongs to them, then also
    corporate).
  • I have updated the documentation, if applicable.
  • I have ensured that the change is tested somewhere in the testsuite
    (adding new test cases if necessary).
  • My code follows the prevailing code style of this project.

Scratch memory allocation in ImageOutput::to_native_rectangle was using
the number of channels squared.
Copy link
Collaborator

@lgritz lgritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holy crap! That's an embarrassing one. Good catch!

I will merge as soon as the CI finishes.

@lgritz lgritz merged commit 4f97ff6 into AcademySoftwareFoundation:master Nov 11, 2021
lgritz pushed a commit to lgritz/OpenImageIO that referenced this pull request Nov 11, 2021
…cademySoftwareFoundation#3176)

Scratch memory allocation in ImageOutput::to_native_rectangle was using
the number of channels squared.
lgritz pushed a commit to lgritz/OpenImageIO that referenced this pull request Nov 11, 2021
…cademySoftwareFoundation#3176)

Scratch memory allocation in ImageOutput::to_native_rectangle was using
the number of channels squared.
@brechtvl brechtvl deleted the fix-excessive-scatch-mem branch May 27, 2023 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants