Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Update cuDF to always explicitly specify RMM pool size and avoid using rmm::detail APIs #14658

Closed
Tracked by #10
harrism opened this issue Dec 20, 2023 · 0 comments · Fixed by #14741
Closed
Tracked by #10

[BUG] Update cuDF to always explicitly specify RMM pool size and avoid using rmm::detail APIs #14658

harrism opened this issue Dec 20, 2023 · 0 comments · Fixed by #14741
Assignees
@harrism
Labels
bug Something isn't working libcudf Affects libcudf (C++/CUDA) code.

Comments

@harrism
Copy link
Member

harrism commented Dec 20, 2023 •
edited

rapidsai/rmm#1417 and followon PRs make a number of improvements and changes to help consumers of RMM avoid accessing RMM's detail namespace.

  • initial pool size in pool_memory_resource is required to be provided (no longer optional)
  • rmm::detail::available_device_memory() (moving to `rmm::available_device_memory())
  • Move alignment utility functions out of detail namespace.
  • Add utility to calculate an aligned percentage of free device memory in bytes. This way, existing instantiations of pool_memory_resource(upstream) can be replaced by pool_memory_resource(upstream, rmm::percent_of_free_device_memory(50), which matches previous behavior.

cuDF should be updated for these changes once rapidsai/rmm#1417 is merged. I have a PR ready to open.
@harrism harrism added bug Something isn't working Needs Triage Need team to review and classify labels Dec 20, 2023
@harrism harrism self-assigned this Dec 20, 2023
@harrism harrism added the libcudf Affects libcudf (C++/CUDA) code. label Dec 20, 2023
@harrism harrism changed the title [BUG] cuDF calls rmm::detail::available_device_memory [BUG] Update cuDF to always explicitly specify RMM pool size and avoid using rmm::detail APIs Jan 11, 2024
This was referenced Jan 11, 2024
Closed
Merged
rapids-bot bot pushed a commit that referenced this issue Jan 17, 2024
@harrism
Provide explicit pool size and avoid RMM detail APIs (#14741)
8f5e64d 
This PR fixes up cuDF to avoid usage that will soon be deprecated in RMM.

Depends on rapidsai/rmm#1417

Fixes #14658

Authors:
  - Mark Harris (https://github.com/harrism)
  - Yunsong Wang (https://github.com/PointKernel)
  - Nghia Truong (https://github.com/ttnghia)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - Yunsong Wang (https://github.com/PointKernel)

URL: #14741
@bdice bdice removed the Needs Triage Need team to review and classify label Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@harrism harrism

Labels
bug Something isn't working libcudf Affects libcudf (C++/CUDA) code.
Projects
cuDF/Dask/Numba/UCX
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Provide explicit pool size and avoid RMM detail APIs harrism/cudf
2 participants
@harrism @bdice