Are there any plans for adding GPU/CUDA support for some functions?

[din14970]

This is not a bug, more a feature request and question. I deal with large datasets of thousands of images that all need to be transformed to polar coordinates. At 5 - 200 ms per image, depending on the image size, this can really add up. A lot of image processing routines perform well when parallelized on the GPU, but the skimage.transform functions don't seem to work on cupy arrays. Since I couldn't find a convenient polar transform function that works on the GPU elsewhere, I threw together the following to emulate the skimage.transform.warp_polar function (though it doesn't support all kwargs):

import cupy as cp
import cupyx.scipy.ndimage as ndigpu

def warp_polar_gpu(image, center=None, radius=None, output_shape=None, **kwargs):
    if radius is None:
        radius = int(np.ceil(np.sqrt((image.shape[0] / 2)**2 + (image.shape[1] / 2)**2)))
    cx, cy = image.shape[1] // 2, image.shape[0] // 2
    if center is not None:
        cx, cy = center
    if output_shape is None:
        output_shape = (360, radius)
    delta_theta = 360 / output_shape[0]
    delta_r = radius / output_shape[1]
    t = cp.arange(output_shape[0])
    r = cp.arange(output_shape[1])
    R, T = cp.meshgrid(r, t)
    X = R * delta_r * cp.cos(cp.deg2rad(T * delta_theta)) + cx
    Y = R * delta_r * cp.sin(cp.deg2rad(T * delta_theta)) + cy
    coordinates = cp.stack([Y, X])
    polar = ndigpu.map_coordinates(image, coordinates, order=1)
    return polar

This is probably not an optimal implementation. But not counting CPU - GPU data transfer time (image and polar are both cupy arrays), on a Tesla V100 this works 14 times faster than warp_polar on a 256x256 image (5 ms vs 0.35 ms) and 360 times faster for a 4000x4000 image (180 ms vs 0.5 ms). Obviously the pay-off gets better as the images get larger.

Are there any plans to add GPU support for some scikit-image functions? Would there be any interest in including implementations like those above to the project, and if so, where should one contribute them?

[grlee77]

Hi @din14970, thank you for the interest. We are indeed interested in this and have some recent grant funding from the Chan Zuckerberg Institute (CZI) to prototype methods for providing different backends for scikit-image (e.g. CuPy). I don't have a new PR here yet for that and we are still considering how best to approach it. One recent thread of discussion is enabling upstream SciPy's ndimage module itself to dispatch on CuPy and Dask arrays (see scipy/scipy#10204). That would get some things working here, but we would still need independent GPU implementations for scikit-image code that uses Cython rather than NumPy or SciPy operations, though.

If you are not already doing so, I would encourage you to try out the 9.0 branch of CuPy as it has a lot more things in cupyx.scipy.ndimage implemented as compared to the 8.x releases. The final CuPy 9.0.0 release is scheduled for late April.

Last year, I had worked on CuPy-based implementations of a decent subset of scikit-image at https://github.com/mritools/cupyimg. That repository is currently being migrated to a new home with better testing/support. The new repository will be made public later in April (I will try to remember and ping you when it is). It will also be open source and welcome community contributions.

The existing public cupyimg repository linked above should work for the latest CuPy 9.0.0b3 release and warp polar is implemented there in a similar fashion to your example. cupyimg is likely not still fully compatible with CuPy 8.x releases, though.

[din14970]

Thanks for the links! Indeed it might be better to get scipy working on CuPy and Dask arrays than to add a number of ad-hoc GPU-compatible re-implementations to scikit-image, though your cupyimg project is already a decent start. I'll wait to see how those things evolve and for now continue with my custom function. CuPy 9.0 also looks really good, thanks for the tip.

[grlee77]

The new repository will be made public later in April (I will try to remember and ping you when it is). It will also be open source and welcome community contributions.

The repo I was referring to here is now available as a new RAPIDS project called cuCIM. Currently, it should be possible to install for CUDA 11.x via conda from the rapidsai-nightly channel. For example, to create a new conda environment with it:

conda create -n cucim -c rapidsai-nightly -c conda-forge/label/cupy_rc -c conda-forge cucim
conda activate cucim

In a few days it should be ready on the main rapidsai channel as well. If you try it and encounter any problems, please let us know over at the cuCIM issues page.

One known issue at the moment is that cuCIM requires a newer version of CuPy than other RAPIDS projects. For a future RAPIDS 0.20 release, I think the whole RAPIDS ecosystem will have updated to CuPy 9.0, making things easier to install together in the same environment.

[lmmx]

Just wanted to add in links to the presentation at GTC and cuCIM roadmap, exciting stuff! Worth noting that for now only TIFF is supported and common formats like PNG/JPEG are coming in the next version

[jakirkham]

<shameless plug>

There are also 2 blogposts: one from NVIDIA and one from Quansight

</shameless plug>

[grlee77]

deleted the discussion as we are moving back to issues. will leave this closed as being answered:

we plan to add backend support so third-party libraries like dask-image or cuCIM can be used, but will not be hosting GPU code within scikit-image itself