Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Convert rmm resource parameters to rmm::device_async_resource_ref #1979

Open
ttnghia opened this issue Apr 18, 2024 · 2 comments
Open
Assignees

Comments

@ttnghia
Copy link
Collaborator

ttnghia commented Apr 18, 2024

In order to move along with rmm code refactoring, similar to rapidsai/cudf#15507, we need to perform a repository-wide conversion from rmm::mr::device_memory_resource * into rmm::device_async_resource_ref.

This needs to be done ASAP before rmm::mr::device_memory_resource * is removed from rmm.

@pmattione-nvidia
Copy link
Collaborator

Merged PR 2011 does much of this work; this conversion has now been performed at all of the kernel interfaces. However the resource allocator in SparkResourceAdaptor.jni still uses device_memory_resource and needs to be updated.

@pmattione-nvidia
Copy link
Collaborator

pmattione-nvidia commented Jun 11, 2024

Further work is blocked until RMM updates its memory resource classes. The cuDF Java initialize() function ultimately creates the RMM C++ memory resources, but they all still inherit from device_memory_resource. The RMM team says that they still need to figure some things out before they can update them to use the new memory concepts.

In anticipation of the upcoming changes, the following draft PRs have been created, containing all of the work that can be done until then:

cuDF Java Code Draft PR
spark-rapids-jni Draft PR

The remaining work that needs to be done before we can update our Java code is being tracked here:

Refactor RMM resources issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants