DOC: Document CUDA acceleration

[thewtex]

No description provided.

[thewtex]

Hi John @jakirkham et al!

Do you have any suggestions on the following dask/cupy/cucim/kvikio issues or do you know who would?

1. We currently get the traceback:

  File "/home/matt/src/ngff-zarr/ngff_zarr/cli.py", line 62, in _multiscales_to_ngff_zarr                                                       
    to_ngff_zarr(output_store, multiscales, progress=rich_dask_progress)                                                                        
  File "/home/matt/src/ngff-zarr/ngff_zarr/to_ngff_zarr.py", line 253, in to_ngff_zarr                                                          
    dask.array.to_zarr(                                                 
  File "/home/matt/bin/micromamba/envs/ngff-zarr/lib/python3.10/site-packages/dask/array/core.py", line 3730, in to_zarr
    return arr.store(z, lock=False, compute=compute, return_stored=return_stored)
[...]
  File "/home/matt/bin/micromamba/envs/ngff-zarr/lib/python3.10/site-packages/zarr/core.py", line 2308, in _chunk_setitem_nosync
    cdata = self._process_for_setitem(ckey, chunk_selection, value, fields=fields)
  File "/home/matt/bin/micromamba/envs/ngff-zarr/lib/python3.10/site-packages/zarr/core.py", line 2374, in _process_for_setitem
    chunk[chunk_selection] = value                                                                                                              
  File "cupy/_core/core.pyx", line 1475, in cupy._core.core._ndarray_base.__array__
TypeError: Implicit conversion to a NumPy array is not allowed. Please use `.get()` to construct a NumPy array explicitly.

Is there an elegant way to address this?

2. I took the install commands from the cucim docs. Is this current best practice and will there simplifications in the future?

3. Can we easily also add kvikio support when reading / writing?

[thewtex]

cudatoolkit= -> cuda-version=<CUDA VERSION>

per @jakirkham 's recommendation.

[thewtex]

Thanks for the pointers, @jakirkham! Now working with cucim + blosc and cucim + kvikio 🎇

In the kvikio zarr notebook, it mentions:

Some algorithms, such as LZ4, can be used interchangeably on CPU and GPU but Zarr will always use the compressor used to write the Zarr file. We are working with the Zarr team to fix this shortcoming but for now, we will use a workaround where we patch the metadata manually.

Do you know if there is an issue, etc. to track progress on this?

[thewtex]

In the kvikio zarr notebook, it mentions:

Some algorithms, such as LZ4, can be used interchangeably on CPU and GPU but Zarr will always use the compressor used to write the Zarr file. We are working with the Zarr team to fix this shortcoming but for now, we will use a workaround where we patch the metadata manually.

Do you know if there is an issue, etc. to track progress on this?

@madsbk @ivirshup I now see the discussion here: rapidsai/kvikio#261

I think it's important to show if data can be read both direct to GPU by this library, and in an environment without the NVIDIA libraries or a GPU.

@ivirshup yes! This is an effort to progress towards this goal -- something that works anywhere, and, if you have the NVIDIA libraries and a GPU, you can get an accelerated version.

I see a link to zarr-developers/numcodecs#316 . Is that where harmonious blosc / kvikio zarr .zarray metadata is being discussed? Or is there something else? CC @joshmoore

[thewtex]

Do you know if there is an issue, etc. to track progress on this?

Tracking this now in this repository in #70