[FEA]: NCCL device API

[BDHU]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

General cuda-python

Is your feature request related to a problem? Please describe.

I was playing around with NCCL device-initiated API and I wonder how I can support CUDA kernels that include nccl_device.h. The problem seems to be that nccl_device.h also imports stdint.h which has include_next, which is a GCC-specific preprocessor directive used by the system stdint.h. Any pointers? Thanks.

Describe the solution you'd like

Support arbitrary third-party headers in CUDA kernel

Describe alternatives you've considered

No response

Additional context

No response

[leofang]

Hi @BDHU, please kindly report this issue to the https://github.com/NVIDIA/nccl repo. The NCCL team hangs out there 🙂

Based on the experience we have with CUDA C++ (aka https://github.com/NVIDIA/cccl; note that CCCL != NCCL), having system headers showing up in the device path might have some challenges, especially when compiling using NVRTC. Depending on the need it might require some delicate surgery and refactoring on the library side. cc @xiakun-lu for vis