wrapped function option

[Sh4zKh4n]

I am wondering if there could be a function thats generic to allow wrapping other imaging libraries?

I have had problems in porting over into dask scikit image functions and ndimage functions over. Though when I look at the code for dask image, one of the functions isnt there and looks like you have a util file to wrap the ndimage.median filter to suprress certain attributes. I wonder if there is an option to make this more widely usable or documented. I think ive even seen some code by @jni in a presentation which uses dask_image.utils but the docs have no information on the utils. One of the reasons I ask is that using some of the other functions can cause memory issues and I wanter if this is to do with the output arg and the util looks like it suppresses that and adds some common args that fit into these types of libraries.

I have tried map_blocks and overlay and bpth seem to have expanding memory issues when it comes to arrays bigger than memory. Especially with bigger than memory arrays.

as an example of a func I write for a 3d array function for map blocks I ways do something like this

def func(img, args)
img2 = library.func(img, args)
return img2

out = func(img,args)

out.to_zarr('filepath.zar', ovewrite=True)

I often have memory issues and nothign is written to file. It is intialised but the it gets stuck between memory and the hard disc and not to the file. So wondered if this could be documented to allow others to use? Unless theres another option?

[GenevieveBuckley]

Hi @Sh4zKh4n

The generic wrapping functions available for dask arrays/images are dask.array.map_overlap and dask.array.map_blocks. As you've found though, it's not always straightforward! Sometimes the internal implementations of functions aren't easily compatible with dask arrays (one of the most likely reasons you don't find every scipy ndimage function supported here). For that reason there's no truly generic wrapping function that would work out of the box for all things.

I'm not sure I saw the presentation of Juan's you are talking about here. dask-image doesn't have a single utils module, but it does contain utility functions within each module. Those aren't part of our public API, in general we don't expect users to work with any of those directly and that's why you don't see those in the docs. I don't know what the talk was about so I can't really guess what utility function it might have referred to.

If you're running into memory issues, I suspect you might want to reduce the size of your chunks. There are some best practices docs for dask arrays, and for dask in general you might like to check out. With those things in mind, then you can do a little trial and error to find out what works for you.

[GenevieveBuckley]

Here's a table of the coverage of dask-image compared with the scipy ndimage functions, which may be useful:

Function name	SciPy ndimage	dask-image
affine_transform	X
binary_closing	X	X
binary_dilation	X	X
binary_erosion	X	X
binary_fill_holes	X
binary_hit_or_miss	X
binary_opening	X	X
binary_propagation	X
black_tophat	X
center_of_mass	X	X
convolve	X	X
convolve1d	X
correlate	X	X
correlate1d	X
distance_transform_bf	X
distance_transform_cdt	X
distance_transform_edt	X
extrema	X	X
find_objects	X
fourier_ellipsoid	X
fourier_gaussian	X	X
fourier_shift	X	X
fourier_uniform	X	X
gaussian_filter	X	X
gaussian_filter1d	X
gaussian_gradient_magnitude	X	X
gaussian_laplace	X	X
generate_binary_structure	X
generic_filter	X	X
generic_filter1d	X
generic_gradient_magnitude	X
generic_laplace	X
geometric_transform	X
grey_closing	X
grey_dilation	X
grey_erosion	X
grey_opening	X
histogram	X	X
imread	X	X
iterate_structure	X
label	X	X
labeled_comprehension	X	X
laplace	X	X
map_coordinates	X
maximum	X	X
maximum_filter	X	X
maximum_filter1d	X
maximum_position	X	X
mean	X	X
median	X	X
median_filter	X	X
minimum	X	X
minimum_filter	X	X
minimum_filter1d	X
minimum_position	X	X
morphological_gradient	X
morphological_laplace	X
percentile_filter	X	X
prewitt	X	X
rank_filter	X	X
rotate	X
shift	X
sobel	X	X
spline_filter	X
spline_filter1d	X
standard_deviation	X	X
sum	X	X
uniform_filter	X	X
uniform_filter1d	X
variance	X	X
watershed_ift	X
white_tophat	X
zoom	X

[GenevieveBuckley]

Since this is a feature request for something we're not able to implement I'm going to mark this issue as closed, but I hope that the information I've given above is helpful to you.

[grlee77]

I think ive even seen some code by @jni in a presentation which uses dask_image.utils but the docs have no information on the utils

Do you maybe mean skimage.util.apply_parallel? That does use dask.map_overlap internally. It is still pending a documentation example (scikit-image/scikit-image#4214) and better color image support (scikit-image/scikit-image#4927), but hopefully those will be included in the next release.