Skip to content

Update HPC documentation page#663

Merged
jayavenkatesh19 merged 14 commits into
rapidsai:mainfrom
jayavenkatesh19:hpc-documentation
May 13, 2026
Merged

Update HPC documentation page#663
jayavenkatesh19 merged 14 commits into
rapidsai:mainfrom
jayavenkatesh19:hpc-documentation

Conversation

@jayavenkatesh19
Copy link
Copy Markdown
Contributor

@jayavenkatesh19 jayavenkatesh19 commented Feb 11, 2026

Towards #54

Replaces the legacy HPC page (outdated Dask with CUDA 11.0.3 content) with a modern guide covering single-GPU RAPIDS deployment on SLURM clusters.

Consists of the following sections:

  • SLURM basics: Partitions, interactive vs bash jobs, srun/sinfo/sbatch flags, session persistence with tmux
  • Lmod RAPIDS module creation
  • Apptainer/Enroot+Pyxis RAPIDS containers
  • Single GPU cudf.pandas example.

Multi-GPU workloads, cloud SLURM (Coreweave) will be added to this page in further iterations.

Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
@jayavenkatesh19 jayavenkatesh19 self-assigned this Feb 11, 2026
@jayavenkatesh19 jayavenkatesh19 added improvement Improves an existing functionality hpc General HPC related labels Feb 11, 2026
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
@jayavenkatesh19 jayavenkatesh19 marked this pull request as ready for review May 6, 2026 17:29
@jayavenkatesh19 jayavenkatesh19 requested a review from a team as a code owner May 6, 2026 17:29
Copy link
Copy Markdown
Contributor

@ncclementi ncclementi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great, I left some comments to discuss and improve some things.

I wonder if in a follow up we should try to get an example that uses multi-gpu, maybe with cudf polars or something.

Comment thread source/hpc.md Outdated
Comment thread source/hpc.md Outdated
Comment thread source/hpc.md Outdated
Comment thread source/hpc.md Outdated
Comment thread source/hpc.md
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
Copy link
Copy Markdown
Contributor

@ncclementi ncclementi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a heads up I did not test it directly. But everything seems reasonable. We can iterate over if we need to in the future

@jayavenkatesh19 jayavenkatesh19 merged commit 9c97c77 into rapidsai:main May 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hpc General HPC related improvement Improves an existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants