Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory leak in libjpeg-turbo decoder implementation in case of corrupted images #4138

Merged
merged 4 commits into from
Aug 5, 2022

Conversation

jantonguirao
Copy link
Contributor

@jantonguirao jantonguirao commented Aug 4, 2022

Signed-off-by: Joaquin Anton janton@nvidia.com

Category:

Bug fix

Description:

  • Fixes the memory leak reported by pipe rebuild has a memory leak? #4076
  • The issue came from throwing exceptions in code where memory management was handled manually
  • The solution is to use unique_ptr's to handle the memory, and .release() on the successful path.
  • Removed dead code

Additional information:

Affected modules and functionalities:

fn.decoders.image

Key points relevant for the review:

Replacement of manual memory handling with unique_ptr

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A
    Checked memory consumption locally with the dataset provided in the mentioned issue

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-2944

…rrupted images

Signed-off-by: Joaquin Anton <janton@nvidia.com>
return nullptr;
}

jpeg_create_decompress(&cinfo);
auto destroy_cinfo = AtScopeExit([&cinfo]() {
jpeg_destroy_decompress(&cinfo);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is crucial. Some paths ended up without calling jpeg_destroy_decompress which was the source of the leak

@@ -546,23 +524,6 @@ uint8* Uncompress(const void* srcdata, int datasize,
return dstdata;
}

uint8* Uncompress(const void* srcdata, int datasize,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

@@ -606,196 +567,5 @@ bool GetImageInfo(const void* srcdata, int datasize, int* width, int* height,
return true;
}

// -----------------------------------------------------------------------------
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

FewerArgsForCompiler argball(datasize, flags, nwarn,
std::move(allocate_output));
const UncompressFlags& flags) {
FewerArgsForCompiler argball(datasize, flags);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess now we could get rid of that, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used to keep some state too, apparently. So I'll leave it for now.

dali/imgcodec/decoders/jpeg/jpeg_mem.cc Show resolved Hide resolved
FewerArgsForCompiler argball(datasize, flags, nwarn,
std::move(allocate_output));
const UncompressFlags& flags) {
FewerArgsForCompiler argball(datasize, flags);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used to keep some state too, apparently. So I'll leave it for now.

Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Copy link
Contributor Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5548261]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5548261]: BUILD FAILED

Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Copy link
Contributor Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5550334]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5550334]: BUILD PASSED

FewerArgsForCompiler argball(datasize, flags, nwarn,
std::move(allocate_output));
const UncompressFlags& flags) {
FewerArgsForCompiler argball(datasize, flags);
uint8* const dstdata = UncompressLow(srcdata, &argball);
Copy link
Member

@stiepan stiepan Aug 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So here we releasded the ownership of the memory and the caller of the function must take care of releasing it, right? As a food for thought, maybe this function should also return smart pointer? It looks a bit brittle.

Signed-off-by: Joaquin Anton <janton@nvidia.com>
@jantonguirao
Copy link
Contributor Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5557749]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5557749]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [5557749]: BUILD PASSED

@jantonguirao jantonguirao merged commit 63c7648 into NVIDIA:main Aug 5, 2022
jantonguirao added a commit to staniewzki/DALI that referenced this pull request Aug 5, 2022
…rrupted images (NVIDIA#4138)

Signed-off-by: Joaquin Anton <janton@nvidia.com>
stiepan pushed a commit that referenced this pull request Aug 18, 2022
…rrupted images (#4138)

Signed-off-by: Joaquin Anton <janton@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants