Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reader chunk_dims are too coarse-grained for dask #33

Open
toloudis opened this issue Jan 23, 2024 · 0 comments
Open

reader chunk_dims are too coarse-grained for dask #33

toloudis opened this issue Jan 23, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@toloudis
Copy link
Contributor

toloudis commented Jan 23, 2024

Feature Description

Allow more specificity in requested chunk_dims for get_image_dask_data (_read_delayed).

Use Case

Dask best practices for chunk sizing: https://docs.dask.org/en/latest/delayed-best-practices.html
Currently bioio only allows you to choose a dimension AXIS for chunking and always chunks with the whole range of that dimension.
When reading a large file with something like t=500, c=2, z=150, y=1000, x=2000, (each xy slice is about 4MB) we don't really have the option to chunk by just a few, or even half, of the z slices.

Solution

Not sure about the api to use here, but possibly pass in a chunk_size with actual numeric values.
So the user would probably do this, as an example:

im = BioImage(path)
dims = im.dims
im.get_image_dask_data(chunk_size=[1,1,1,dims.Y*0.5, dims.X*0.5]). # (Defaults to something sensible?)
@toloudis toloudis added the enhancement New feature or request label Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant