Building CuPy with PTDS enabled

[jakirkham]

Is it possible to build CuPy with PTDS enabled? What issues (if any) would one encounter when trying this?

ref: https://developer.nvidia.com/blog/gpu-pro-tip-cuda-7-streams-simplify-concurrency/

[kmaehashi]

Hint for future readers: PTDS = per-thread default stream

Haven't tested but it seems conflicting with CuPy's default stream mechanism?

[jakirkham]

Is it possible to override generally what stream CuPy uses?

[leofang]

I guess if we can locate all places that default the stream to 0 and set the Stream constructor's default to PTDS, we might have a chance to make it work. The get_current_stream() mechanism we have does not seem to prevent the change from happening. But, I'd hope this to be an opt-in change, which means we will have a headache when packaging? It seems we will need two versions of wheels (without and with PTDS), am I right?

[leofang]

ref: numba/numba#5137

[kmaehashi]

But, I'd hope this to be an opt-in change, which means we will have a headache when packaging? It seems we will need two versions of wheels (without and with PTDS), am I right?

Right for thrust and cub, but other kernels are compiled at runtime using NVRTC but it seems it does not support PTDS option.

[leofang]

Right for thrust and cub

I thought we also need to pass -DCUDA_API_PER_THREAD_DEFAULT_STREAM to other Python modules even when being compiled by the host compiler? Especially for driver and runtime?

but other kernels are compiled at runtime using NVRTC but it seems it does not support PTDS option.

Good point...Would be nice to get a confirmation for this.

[jakirkham]

I asked offline and was told PTDS should work with NVRTC.

https://docs.nvidia.com/cuda/cuda-driver-api/stream-sync-behavior.html#stream-sync-behavior

[jakirkham]

cc @pentschev (for vis)

[leofang]

@jakirkham Can this be closed now since #4322 is merged?