[BUG] Segfault when a device_buffer is released as its memory resource has already been released

[viclafargue]

Describe the bug
A segfault is triggered when a device_buffer is released as its memory resource has already been released.

Steps/Code to reproduce bug

1. Enable logging (or any UpstreamResourceAdaptor)
2. Allocate some memory through native code.
3. Disable logging (or remove the UpstreamResourceAdaptor)

Explanation :
A pointer to the logging_resource_adaptor will be saved in the device_buffer instance. However, whereas RMM's Python classes retain a reference to the memory resource preventing its deletion, the native code doesn't. Because of this, the logging_resource_adaptor can be released before the device_buffer is, making its proper release impossible.

Expected behavior
No segfault should be observed.

Note:
Collecting the garbage collector before disabling the RMM logging seems to fix the issue. However, it doesn't look like a very good long term solution.

[shwina]

Sync'd offline, and it looks like this issue arises for a custom extension type that is composed of a C++ type, which in turn holds a reference to a rmm::device_uvector. That device_uvector depends on the memory resource being alive at the time of deallocation. The simplest solution here is for the custom extension type to hold a reference to the current memory resource, in the same way that rmm.DeviceBuffer does.

[harrism]

So is this a bug that needs to be fixed in RMM, or in @viclafargue 's code?

[jakirkham]

I thought this was fixed with PR ( #931 )?

[harrism]

@shwina can you confirm?

[shwina]

@viclafargue does the solution I posted above work for your use case? #931 fixed a somewhat different issue than this one.

[viclafargue]

Yes, #931 is a different issue. As discussed offline, the lifetime of the memory resource should be the responsibility of the user. Holding a reference to the current memory resource in the same way the rmm.DeviceBuffer does should indeed solve the issue. Thank you for the answers. Closing the issue.