Set up CUDA and pycuda on DCC

Creating the environment

Install conda/miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh

Create new conda environment

conda create -n gpu -c conda-forge pycuda python=3

Activate new environment

conda activate gpu

Install the rest of CUDA

conda install -c nvidia cudatoolkit
conda install -c nvidia cuda-libraries-dev
conda install -c nvidia cuda-nvcc
conda install -c nvidia cuda-gdb
# conda install -c conda-forge pycuda  # pycuda is required, but should have been installed when the env was created

Running code in the environment

Get to interactive GPU node

srun -p gpu-common --gres=gpu:1 --pty bash -i

Run directly

srun -p gpu-common --gres=gpu:1 [command to run]

All standard srun arguments, like --mem, apply. According to https://dcc.duke.edu/dcc/slurm/#gpu-jobs, as of 10/1/2022 you can specify a specific GPU type and count.