Defining the DpnpNdArray type so that Numba properly caches dpjit functions

[diptorupd]

The compute follows data requires that the execution of a kernel takes place on the device where the data was allocated. The question is is using the device filter string enough to decide where execution takes place or do we need to enforce the restriction on the actual queue?

[AlexanderKalistratov]

It's not the question "Should filter string or SyclQueue be used for compute follows data?" it is question "what information we should use to dispatch numba-dpex functions". And it is not related to dpctl actually.

As I said here #945 (comment) Device information must be part of this dispatch mechanics. I.e. be part of type signature. Either DpnpArray or SyclQueue or SyclDevice. We have options. Also it is not necessary to be filter_string. We have options.
filter_string is just most human friendly and straight-forward way.

Moreover, just to be clear are you fully aware what a filter string is? Can you explain why you need device rank?

I don't need device rank itself. I need clear way to differ one device from another. Since filter_string is unique for the device it is meets the requirement.

[diptorupd]

I feel we are talking about the same thing.

There is no disagreement that device information is required to correctly compile and dispatch. We are already doing that by extracting the information from the queue. The current dispatcher does that by only looking at the queue.sycl_device.backend attribute and verifying that that back end is opencl or level_zero. Since, the compiler is not yet generating different SPIR-V for different types of device, we are not looking at queue.sycl_device.device_type. Once we start doing that, we can change the code generator, all the required device information and attributes is accessible from the queue.sycl_device object.

You want some way to manually inspect the device information from the type signature, right?

We can add the queue.sycl_device.backend and queue.sycl_device.device_type values to the type signature for that purpose.

[AlexanderKalistratov]

Actually you are dispatching on filter_string:
https://github.com/IntelPython/numba-dpex/blob/main/numba_dpex/core/types/usm_ndarray_type.py#L203 - self.device is filter string.

We can add the queue.sycl_device.backend and queue.sycl_device.device_type values to the type signature for that purpose.

You need to add these values to the type signature. Also you need to add some kind of device number/id/guid or capabilities. To differ for example discrete gpu from integrated one. Or one discrete from another. Together these three values forms something very close to filter_string.
As I said, you could reinvent it. But it is much better to take it from dpctl.

Since, the compiler is not yet generating different SPIR-V for different types of device

It is not only about SPIR-V. It is about the whole numba function. Since on integrated graphics fp64 is not supported we need to compile two different numba function versions for cpu and integrated gpu or for discrete gpu and integrate gpu.
Right now it would work, since you are dispatching on filter_string. But it wouldn't work if you'll remove this dispatch.

You want some way to manually inspect the device information from the type signature, right?

It would be a nice feature to have. But having proper device information in type signature is the question of correct dispatch.

[diptorupd]

@AlexanderKalistratov based on our conversation I now understand your point.

The issue is not about the kernel decorator, but about dpjit. Also, the implementation of compute follows data using dpctl.SyclQueue is not the point of concern.

The issue you are raising is that the type signature of the DpnpNdArray should be defined such that Numba does not recompile the function just because the queue changed. To address the issue, we need a way to define the key for the DpnpNdArray class that clearly distinguishes between devices.