CUDA: Support for `@overload` #6944

gmarkall · 2021-04-19T16:41:08Z

The implementation consists of:

Addition of get_call_template, nopython_signatures, and get_overload to the class that is now DeviceDispatcher. These are needed by the overload infrastructure, and are inspired by the equivalents in _DispatcherBase.
Registration of a CUDA jit decorator. The registration with dispatcher_register is removed and replaced with this new registration, because it did nothing (it was for the target kwarg to the numba.jit decorator, which is deprecated). The jit decorator for overloads is a bit special in that it always adds the device=True kwarg to the regular jit decorator, because overloads will always be device functions.
Addition of a no_cpython_wrapper kwarg to jitdevice, which is needed because the overload infrastructure passes it. No action is required regardless of its value, because there is no CPython wrapper in the CUDA target.

A refactoring is also made: DeviceFunctionTemplate is renamed to DeviceDispatcher, as it reflects more closely what it is - it is
analagous to the Dispatcher class used for kernels, and the DeviceFunction class is more analagous to the _Kernel class. Eventually Dispatcher will subsume these classes as we move towards a more "modern" dispatcher implementation in the CUDA target.

Tests are added; these are all skipped on the simulator because overloading doesn't really exist as a concept without compilation.

@stuartarchibald

Joint work with @stuartarchibald. The implementation consists of: - Addition of `get_call_template`, `nopython_signatures`, and `get_overload` to the class that is now `DeviceDispatcher`. These are needed by the overload infrastructure, and are inspired by the equivalents in `_DispatcherBase`. - Registration of a CUDA `jit` decorator. The registration with `dispatcher_register` is removed and replaced with this new registration, because it did nothing (it was for the `target` kwarg to the `numba.jit` decorator, which is deprecated). The `jit` decorator for overloads is a bit special in that it always adds the `device=True` kwarg to the regular `jit` decorator, because overloads will always be device functions. - Addition of a `no_cpython_wrapper` kwarg to `jitdevice`, which is needed because the overload infrastructure passes it. No action is required regardless of its value, because there is no CPython wrapper in the CUDA target. A refactoring is also made: `DeviceFunctionTemplate` is renamed to `DeviceDispatcher`, as it reflects more closely what it is - it is analagous to the `Dispatcher` class used for kernels, and the `DeviceFunction` class is more analagous to the `_Kernel` class. Eventually `Dispatcher` will subsume these classes as we move towards a more "modern" dispatcher implementation in the CUDA target. Tests are added; these are all skipped on the simulator because overloading doesn't really exist as a concept without compilation.

stuartarchibald · 2021-04-19T17:09:55Z

This failure: https://dev.azure.com/numba/ff1fe4d0-ed73-4f1c-b894-1d50a27e048f/_apis/build/builds/8536/logs/130 AFAICT is nothing to do with this PR, fixed in #6945

gmarkall · 2021-04-20T09:02:05Z

A quick note on this, mainly for @stuartarchibald: Earlier versions of this had dispatcher_registry[hardware_registry["cuda"]] = Dispatcher in initialize_all(), but I removed it for this patch because it wasn't used for @overload - it will need adding back sometime. My understanding was that this is only needed for the @jit decorator with a hardware / target (whatever we end up calling it) kwarg.

stuartarchibald · 2021-04-21T10:27:54Z

A quick note on this, mainly for @stuartarchibald: Earlier versions of this had dispatcher_registry[hardware_registry["cuda"]] = Dispatcher in initialize_all(), but I removed it for this patch because it wasn't used for @overload - it will need adding back sometime. My understanding was that this is only needed for the @jit decorator with a hardware / target (whatever we end up calling it) kwarg.

Agree, I think the registrations needed to get overload working at present can avoid having a dispatcher registered.

Using prints and capturing stdout to test overloads is both overly complicated and difficult to debug (terminal-based debuggers and debug prints don't work during the tests). The value-based check that replaces it suffers neither of these issues, and it is at least as clear what each test does (more so, in my opinion).

gmarkall · 2021-04-21T13:11:16Z

@stuartarchibald As you suggested OOB I've replaced the printing and captured output in the tests with a value-based check - this is much tidier, and easier to deal with issues.

stuartarchibald

Thanks for the patch @gmarkall, great to see this working :) Once minor comment to resolve else looks good.

stuartarchibald · 2021-04-21T13:49:03Z

numba/cuda/tests/cudapy/test_overload.py

+    def test_call_hardware_overloaded(self):
+        def kernel(x):
+            hardware_overloaded(x)
+
+        expected = CUDA_HARDWARE_OL
+        self.check_overload(kernel, expected)
+
+    def test_generic_calls_hardware_overloaded(self):
+        def kernel(x):
+            generic_calls_hardware_overloaded(x)
+
+        expected = GENERIC_CALLS_HARDWARE_OL * CUDA_HARDWARE_OL
+        self.check_overload(kernel, expected)
+
+    def test_cuda_calls_hardware_overloaded(self):
+        def kernel(x):
+            cuda_calls_hardware_overloaded(x)
+
+        expected = CUDA_CALLS_HARDWARE_OL * CUDA_HARDWARE_OL
+        self.check_overload(kernel, expected)
+
+    def test_hardware_overloaded_calls_hardware_overloaded(self):
+        def kernel(x):
+            hardware_overloaded_calls_hardware_overloaded(x)
+
+        expected = CUDA_HARDWARE_OL_CALLS_HARDWARE_OL * CUDA_HARDWARE_OL
+        self.check_overload(kernel, expected)


In this section, maybe test that the CPU calls would pick up the "generic" variants?

This is checking when there are both generic and CUDA overloads, to ensure the CUDA overloads don't interfere with the generic / CPU target. From PR numba#6944 feedback.

stuartarchibald · 2021-04-22T12:35:23Z

Thanks for adding the extra tests in f1b54d0, this looks good. There's a conflict to resolve against mainline and once done this can be tested on the farm. Thanks again.

gmarkall · 2021-04-22T12:49:34Z

@stuartarchibald Many thanks, conflict resolved.

stuartarchibald

Thanks for getting this working, it's great to see the new API in action!

stuartarchibald · 2021-04-22T12:51:32Z

@stuartarchibald Many thanks, conflict resolved.

Thanks, patch approved.

stuartarchibald · 2021-04-22T12:59:04Z

Buildfarm ID: numba_smoketest_cuda_yaml_49.

stuartarchibald · 2021-04-22T14:44:49Z

Buildfarm ID: numba_smoketest_cuda_yaml_49.

Passed.

stuartarchibald mentioned this pull request Apr 19, 2021

Fix issue with array analysis tests needing scipy. #6945

Merged

stuartarchibald added CUDA CUDA related issue/PR 3 - Ready for Review labels Apr 19, 2021

stuartarchibald added this to the Numba 0.54 RC milestone Apr 19, 2021

stuartarchibald added the Effort - medium Medium size effort needed label Apr 19, 2021

DeviceDispatcher: make _compileinfos overloads

d1cd731

stuartarchibald reviewed Apr 21, 2021

View reviewed changes

stuartarchibald added 4 - Waiting on author Waiting for author to respond to review and removed 3 - Ready for Review labels Apr 21, 2021

Check generic hardware overloads used on CPU

f1b54d0

This is checking when there are both generic and CUDA overloads, to ensure the CUDA overloads don't interfere with the generic / CPU target. From PR numba#6944 feedback.

gmarkall added 4 - Waiting on reviewer Waiting for reviewer to respond to author and removed 4 - Waiting on author Waiting for author to respond to review labels Apr 22, 2021

stuartarchibald added 4 - Waiting on author Waiting for author to respond to review and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Apr 22, 2021

Merge remote-tracking branch 'numba/master' into grm-cuda-overload

2d95068

gmarkall added 4 - Waiting on reviewer Waiting for reviewer to respond to author and removed 4 - Waiting on author Waiting for author to respond to review labels Apr 22, 2021

stuartarchibald approved these changes Apr 22, 2021

View reviewed changes

stuartarchibald added 4 - Waiting on CI Review etc done, waiting for CI to finish Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Apr 22, 2021

sklam merged commit 432588b into numba:master Apr 22, 2021

stuartarchibald mentioned this pull request Apr 23, 2021

@intrinsic not working with CUDA #6970

Closed

2 tasks

gmarkall mentioned this pull request Apr 6, 2022

remove the roadmap from the sphinx based docs #7963

Merged

CUDA: Support for `@overload` #6944

CUDA: Support for `@overload` #6944

gmarkall commented Apr 19, 2021

stuartarchibald commented Apr 19, 2021

gmarkall commented Apr 20, 2021

stuartarchibald commented Apr 21, 2021

gmarkall commented Apr 21, 2021

stuartarchibald left a comment

stuartarchibald Apr 21, 2021

gmarkall Apr 22, 2021

stuartarchibald commented Apr 22, 2021

gmarkall commented Apr 22, 2021

stuartarchibald left a comment

stuartarchibald commented Apr 22, 2021

stuartarchibald commented Apr 22, 2021

stuartarchibald commented Apr 22, 2021

CUDA: Support for @overload #6944

CUDA: Support for @overload #6944

Conversation

gmarkall commented Apr 19, 2021

stuartarchibald commented Apr 19, 2021

gmarkall commented Apr 20, 2021

stuartarchibald commented Apr 21, 2021

gmarkall commented Apr 21, 2021

stuartarchibald left a comment

Choose a reason for hiding this comment

stuartarchibald Apr 21, 2021

Choose a reason for hiding this comment

gmarkall Apr 22, 2021

Choose a reason for hiding this comment

stuartarchibald commented Apr 22, 2021

gmarkall commented Apr 22, 2021

stuartarchibald left a comment

Choose a reason for hiding this comment

stuartarchibald commented Apr 22, 2021

stuartarchibald commented Apr 22, 2021

stuartarchibald commented Apr 22, 2021

CUDA: Support for `@overload` #6944

CUDA: Support for `@overload` #6944