Skip to content

[Multi-GPU Polars] New frontends accept RMM config - #22052

Merged
rapids-bot[bot] merged 4 commits into
NVIDIA:mainfrom
madsbk:new-frontends-accept-rmm-config
Apr 10, 2026
Merged

[Multi-GPU Polars] New frontends accept RMM config#22052
rapids-bot[bot] merged 4 commits into
NVIDIA:mainfrom
madsbk:new-frontends-accept-rmm-config

Conversation

@madsbk

@madsbk madsbk commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Allow SPMDEngine, DaskEngine, and RayEngine to accept a MemoryResourceConfig via engine_options["memory_resource_config"].

This enables declarative RMM memory resource configuration, for example through StreamingOptions, instead of requiring users to manually call rmm.mr.set_current_device_resource() before constructing an engine.

@madsbk madsbk self-assigned this Apr 8, 2026
@madsbk madsbk added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 8, 2026
@github-actions github-actions Bot added Python Affects Python cuDF API. cudf-polars Issues specific to cudf-polars labels Apr 8, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Apr 8, 2026
@madsbk
madsbk force-pushed the new-frontends-accept-rmm-config branch 5 times, most recently from 54f21ac to 5b48890 Compare April 9, 2026 13:53
@madsbk
madsbk force-pushed the new-frontends-accept-rmm-config branch from 5b48890 to be61f32 Compare April 9, 2026 13:53
@madsbk
madsbk marked this pull request as ready for review April 9, 2026 15:35
@madsbk
madsbk requested a review from a team as a code owner April 9, 2026 15:35
@madsbk
madsbk requested review from TomAugspurger and bdice April 9, 2026 15:35
@NVIDIA NVIDIA deleted a comment from copy-pr-bot Bot Apr 9, 2026
Comment thread python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/dask.py Outdated
Comment thread python/cudf_polars/docs/cudf-polars-mp.md Outdated
Comment thread python/cudf_polars/tests/experimental/test_options.py Outdated
Comment thread python/cudf_polars/tests/experimental/test_options.py Outdated
Comment thread python/cudf_polars/tests/experimental/test_options.py Outdated
@madsbk

madsbk commented Apr 10, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit 5c4fd5c into NVIDIA:main Apr 10, 2026
163 of 166 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Apr 10, 2026
@madsbk
madsbk deleted the new-frontends-accept-rmm-config branch April 10, 2026 08:32
shrshi pushed a commit to shrshi/cudf that referenced this pull request May 12, 2026
Allow `SPMDEngine`, `DaskEngine`, and `RayEngine` to accept a `MemoryResourceConfig` via `engine_options["memory_resource_config"]`.

This enables declarative RMM memory resource configuration, for example through `StreamingOptions`, instead of requiring users to manually call `rmm.mr.set_current_device_resource()` before constructing an engine.

Authors:
  - Mads R. B. Kristensen (https://github.com/madsbk)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: NVIDIA#22052
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cudf-polars Issues specific to cudf-polars improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants