GPU and sparse: I/O and performance optimizations

[sk1p]

GPU support is still experimental, and needs to be looked at from a performance perspective. Transfers can generally overlap computation, and unified memory could be used. The tiling scheme should also be negotiated appropriately - this needs to be benchmarked.

Edit by @uellue:

Sparse input data support #1207 also requires re-organizing and optimizing tile shape negotiation and execution plan based on the UDFs that will run, array backend that will be used and resources that are available on different kinds of workers. This issue can be extended to the generic case of optimizing settings for different backends.

[sk1p]

While waiting for cupy to compile (sigh) I had a look at how we could use unified memory for our GPU support. cuSignal has support for this, but it is just a small wrapper around the numba mapped_array function

I still want to benchmark this, but based on this, we could implement GPU-awareness into the I/O backend:

• on initializing the backend impl class, set destination to CPU or GPU
• enabling GPU support also forces the copy mode of the mmap backend
• then we directly decode the data into the shared CPU/GPU buffer
• hopefully the driver is clever enough to transfer the data in an efficient way

If this doesn't pan out, we can think about how to integrate multiple GPU streams and manually manage overlapping of transfers and compute.

[uellue]

Bumping milestone since performance impact can be large for both sparse data and GPU data, and it is growing in importance.

[sk1p]

Part of the equation, including initial benchmarks for transfers: LiberTEM/sparseconverter#20