Custom C++ and CUDA Extensions / multiple GPU issue

[adam-ce]

Hi,
I followed the C++ extension tutorial (https://pytorch.org/tutorials/advanced/cpp_extension.html), but I had issues when using multiple GPUs.

The issue is solved: https://discuss.pytorch.org/t/c-cuda-extension-with-multiple-gpus/91241/6

I'd like to suggest adding the corresponding code lines to the tutorial, so that other people won't have the same problem.

Cheers, Adam

[holly1238]

See pytorch/pytorch#48891 and #1196

[adam-ce]

I'm sorry, but I don't see the resolution of this issue.

• as far as i can see, the tutorial is unchanged (so to my understanding, CUDA extension tutorial is not compatible with distributed data parallel #1196 is also not fixed)
• again, as far as i can see, [CUDAExtension] support all visible cards when building a cudaextension pytorch#48891 is about compilation, not memory errors during runtime.

cheers, Adam

[AlbertZhangHIT]

Hi, I also got the same issue for my custom C++ and CUDA extensions.

The RuntimeError: CUDA error: an illegal memory access was encountered occurs especially when some other process is running some GPUs. Let's say, I have 2 GPUs and there is a network training process (PyTorch) running on GPU 0. If I run my custom CUDA extension by setting device=torch.device('cuda:0'), then the custom CUDA extension runs well while it slows down the other network training process. The CUDA error will occur if I set device=torch.device('cuda:1') for my custom CUDA extension.

I found the solution from https://discuss.pytorch.org/t/c-cuda-extension-with-multiple-gpus/91241/6 does not work for the extension that the cpu extension and cuda extension are wrapped up by a common wrapper. Since the compilation of cpu extension will encounter the error that requires CUDA. Does anyone have idea about this issue?