-
-
Notifications
You must be signed in to change notification settings - Fork 807
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cudaLaunchHostFunc
#4338
Add cudaLaunchHostFunc
#4338
Conversation
if not cuda.runtime.is_hip: | ||
stream = cuda.Stream.null | ||
else: | ||
# adding callbacks to the null stream in HIP would segfault... | ||
stream = cuda.Stream() | ||
stream = self.stream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fixes the mistake I made in #3835 🤦🏻♂️ self.stream
was totally ignored...
/test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me overall, I have a few questions only, but thanks for working on it @leofang !
callback (function): Callback function. It must take only one | ||
argument (user data object), and returns nothing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly a naive question, but add_callback
supports/requires 3 arguments: stream, error status and user data. Shouldn't it be the same here? I think for the test you proposed in #4322 (comment) it would be very useful to have the stream so that we can verify that it's indeed happening on the stream we're expecting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I am following the requirement from cudaLaunchHostFunc
that the host function should only have 1 argument instead of 3.
For the test we would like to do there, we could either use add_callback
+ a 3-arg callback, or use this new function launch_host_func
+ a 1-arg callback, with the stream added as part of the arg:
data = []
data.append(stream)
stream.launch_host_func(callback, data)
and check stream.ptr
in the callback function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, I missed the fact that's how cudaLaunchHostFunc
should behave out of my ignorance of that function, thanks for clarifying!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No worries, Peter! I didn't know this function before either. I became aware of it only because your PTDS PR made me wonder if it's possible to test it programmatically other than eyeballing nvvp
/nsys
, and I started browsing the Runtime API doc 😄
stream.launch_host_func( | ||
lambda t: out.append(t[0]), (i, numpy_array)) | ||
|
||
stream.synchronize() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this test here be performed on an explicit stream too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is in fact done in the other test added below. It's just that the stream pointer is wrapped by ExternalStream
there, but it should achieve what you had in mind.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right, I should have checked more than what GH showed to see how the stream was created, thanks for pointing that out and sorry for the false alarm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for checking! It's always good to have extra pairs of eyes 🙏
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @leofang !
pfnCI, test this please. |
Jenkins CI test (for commit d7dc7d3, target branch master) failed with status FAILURE. |
if _is_hip_environment: | ||
raise RuntimeError('This feature is not supported on HIP') | ||
if CUDA_VERSION < 10000: | ||
raise RuntimeError('This feature is only supported on CUDA 10.0+') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@takagi by inspecting the generated C file I noticed Cython (at least 0.29.21 that I'm using) has a nice property that for the if
conditions that can be determined at compile time, Cython will be smart enough to determine the rest of code is dead and eliminate it! (So for RTD and HIP, this function actually stops right after raising.) As a result, it's fine even if we don't add any stubs (for example I forgot to add stubs for CUDA 9.2 😂) Can we rely on this behavior?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds so nice! Would you check if it works with Cython 0.28.0 as, currently, CuPy requires Cython 0.28.0 or later to build it from its source?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @takagi, I just checked Cython 0.28.0 will also eliminate the dead code! This is the warning thrown during cythonizing (in both versions):
[50/59] Cythonizing cupy_backends/cuda/api/runtime.pyx
warning: cupy_backends/cuda/api/runtime.pyx:832:16: Unreachable code
Though I think 0.28 is too outdated and should be avoided as much as possible (#4148).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I ask you to post a new issue so other core members can find that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Jenkins, test this please |
Jenkins CI test (for commit d7dc7d3, target branch master) succeeded! |
LGTM! |
Thanks @takagi @pentschev! |
According to the CUDA Runtime API documentation,
cudaStreamAddCallback
will be deprecated in favor of this new function, added since CUDA 10.0:TODO: