Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in SwissTable::grow_double() #35270

Closed
zifengyu opened this issue Apr 21, 2023 · 4 comments · Fixed by #35347
Closed

Memory leak in SwissTable::grow_double() #35270

zifengyu opened this issue Apr 21, 2023 · 4 comments · Fixed by #35347
Assignees
Milestone

Comments

@zifengyu
Copy link

Describe the bug, including details regarding any error messages, version, and platform.

There are two allocation in grow_double. If the first suceeded but the second failed, the function will return without Free the first buffer

RETURN_NOT_OK(pool_->Allocate(block_size_total_after, &blocks_new));
RETURN_NOT_OK(pool_->Allocate(hashes_size_total_after, &hashes_new_8B));

https://github.com/apache/arrow/blob/main/cpp/src/arrow/compute/key_map.cc#L670

Component(s)

C++

@mapleFU
Copy link
Member

mapleFU commented Apr 22, 2023

I think pool_->Allocate returns a buffer with MemoryManager:

  static std::unique_ptr<PoolBuffer> MakeUnique(MemoryPool* pool, int64_t alignment) {
    std::shared_ptr<MemoryManager> mm;
    if (pool == nullptr) {
      pool = default_memory_pool();
      mm = default_cpu_memory_manager();
    } else {
      mm = CPUDevice::memory_manager(pool);
    }
    return std::make_unique<PoolBuffer>(std::move(mm), pool, alignment);
  }

When PoolBuffer destruct, it will release pool memory:

  ~PoolBuffer() override {
    // Avoid calling pool_->Free if the global pools are destroyed
    // (XXX this will not work with user-defined pools)

    // This can happen if a Future is destructing on one thread while or
    // after memory pools are destructed on the main thread (as there is
    // no guarantee of destructor order between thread/memory pools)
    uint8_t* ptr = mutable_data();
    if (ptr && !global_state.is_finalizing()) {
      pool_->Free(ptr, capacity_, alignment_);
    }
  }

If I'm wrong, please point out that

@zifengyu
Copy link
Author

zifengyu commented Apr 23, 2023

Hi @mapleFU,

SwissTable managers the pool memory directly instead of using PoolBuffer. If the grow_doulbe quit in the middle, the blocks_new is not saved to blocks_, the explict release memory/Free is not called.
https://github.com/apache/arrow/blob/main/cpp/src/arrow/compute/key_map.cc#L770

@mapleFU
Copy link
Member

mapleFU commented Apr 23, 2023

Yes, you're right, seems this may causing memory leak. @westonpace Mind take a look?
@zifengyu Or would you mind give a quick fixing?

@westonpace
Copy link
Member

Thanks for the catch. I think smart pointers should be preferred in general and I've added a PR that switches these two buffers to using Buffer.

pitrou pushed a commit that referenced this issue May 9, 2023
…ls (#35347)

### Rationale for this change

The current code has two storage buffers in the key map which are allocated with MemoryPool::Allocate which does not use smart pointers.  This could have led to a potential memory leak in an OOM scenario where the first allocate fails and it also led to some convoluted code keeping track of the previously allocated size in order to properly call Free.

Furthermore, it seems that this key map could have been getting potentially copied in the swiss join code.  While that was probably not happening (since the copy happened before the key map was initialized) it is still an easy recipe for an accidental double-free later on as we maintain the class.

### What changes are included in this PR?

Those raw buffers are changed to std::shared_ptr<Buffer> to avoid these issues.

### Are these changes tested?

Somewhat, the existing unit tests should ensure we didn't cause a regression.  I didn't introduce a regression test to introduce this potential bug because it would be very difficult to do so.

### Are there any user-facing changes?

No

* Closes: #35270

Authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou pitrou added this to the 13.0.0 milestone May 9, 2023
liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this issue May 11, 2023
…nternals (apache#35347)

### Rationale for this change

The current code has two storage buffers in the key map which are allocated with MemoryPool::Allocate which does not use smart pointers.  This could have led to a potential memory leak in an OOM scenario where the first allocate fails and it also led to some convoluted code keeping track of the previously allocated size in order to properly call Free.

Furthermore, it seems that this key map could have been getting potentially copied in the swiss join code.  While that was probably not happening (since the copy happened before the key map was initialized) it is still an easy recipe for an accidental double-free later on as we maintain the class.

### What changes are included in this PR?

Those raw buffers are changed to std::shared_ptr<Buffer> to avoid these issues.

### Are these changes tested?

Somewhat, the existing unit tests should ensure we didn't cause a regression.  I didn't introduce a regression test to introduce this potential bug because it would be very difficult to do so.

### Are there any user-facing changes?

No

* Closes: apache#35270

Authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
…nternals (apache#35347)

### Rationale for this change

The current code has two storage buffers in the key map which are allocated with MemoryPool::Allocate which does not use smart pointers.  This could have led to a potential memory leak in an OOM scenario where the first allocate fails and it also led to some convoluted code keeping track of the previously allocated size in order to properly call Free.

Furthermore, it seems that this key map could have been getting potentially copied in the swiss join code.  While that was probably not happening (since the copy happened before the key map was initialized) it is still an easy recipe for an accidental double-free later on as we maintain the class.

### What changes are included in this PR?

Those raw buffers are changed to std::shared_ptr<Buffer> to avoid these issues.

### Are these changes tested?

Somewhat, the existing unit tests should ensure we didn't cause a regression.  I didn't introduce a regression test to introduce this potential bug because it would be very difficult to do so.

### Are there any user-facing changes?

No

* Closes: apache#35270

Authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
rtpsw pushed a commit to rtpsw/arrow that referenced this issue May 16, 2023
…nternals (apache#35347)

### Rationale for this change

The current code has two storage buffers in the key map which are allocated with MemoryPool::Allocate which does not use smart pointers.  This could have led to a potential memory leak in an OOM scenario where the first allocate fails and it also led to some convoluted code keeping track of the previously allocated size in order to properly call Free.

Furthermore, it seems that this key map could have been getting potentially copied in the swiss join code.  While that was probably not happening (since the copy happened before the key map was initialized) it is still an easy recipe for an accidental double-free later on as we maintain the class.

### What changes are included in this PR?

Those raw buffers are changed to std::shared_ptr<Buffer> to avoid these issues.

### Are these changes tested?

Somewhat, the existing unit tests should ensure we didn't cause a regression.  I didn't introduce a regression test to introduce this potential bug because it would be very difficult to do so.

### Are there any user-facing changes?

No

* Closes: apache#35270

Authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants