cuda-core: Release GIL when calling `cimport`'d CUDA APIs

[leofang]

[leofang]

If we get this in by v0.4.0, this task is an implementation detail.

If we let it slip out of v0.4.0, this task becomes a bug fix for a regression (cuda-bindings helped us release GIL for all CUDA API calls until #1041).

I have prepared a fix in #1070, so this should be doable.

[kkraus14]

Is the issue that in calling the Python APIs they would explicitly release the GIL before calling the cython functions, but now since we're calling the cython functions directly we are responsible for explicitly releasing the GIL?

[leofang]

Exactly!

[kkraus14]

Would it make sense to include a with nogil: in each Cython function even though they're marked as nogil? Is that something that's allowed? I imagine this could be a bit of a footgun with the Cython bindings.

[leofang]

Copying from offline discussion for the record.

Would it make sense to include a with nogil: in each Cython function even though they're marked as nogil? Is that something that's allowed? I imagine this could be a bit of a footgun with the Cython bindings.

If we change it in the cuda-bindings cy* modules to always do

with nogil:
    call_internal_func()

and the existing callers (ex: cuda-core or RMM) have also released the GIL when calling into cy functions, there incurs a bit of overhead because a few GIL-related CPython calls are generated to ensure the state. I did not measure the delay, though, only inspected the generated code and saw extra calls.