Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support to memmap to take in a list of tiff files with compatible shapes #172

Closed
kushalkolar opened this issue Dec 13, 2022 · 5 comments
Labels
enhancement New feature or request

Comments

@kushalkolar
Copy link

It might be useful to add support to memmap to take in a list of tif files of compatible shapes and return a numpy.memmap with a shape representing concatenation along one of the axes. I could create a wrapper class for my own use cases but I thought I'd post here because this type of functionality might be useful to other users. I could help implement it (no timeline though) if you think it's worth having.

@cgohlke
Copy link
Owner

cgohlke commented Dec 13, 2022

This should already be possible with TiffSequence(...).asarray(out='memmap') but is not available through the memmap function.

@kushalkolar
Copy link
Author

Thanks for the fast response! I just tried this and there is a noticeable memory footprint where it seems to be opening each file in RAM as its iterating through the sequence. By creating a wrapper class I was thinking of mapping slices passed to the wrapper instance __getitem__ to the corresponding memmap from a list of memmaps

@kushalkolar
Copy link
Author

(does this functionality already exists or is of interest to tifffile)

@cgohlke cgohlke added the enhancement New feature or request label Dec 14, 2022
@cgohlke
Copy link
Owner

cgohlke commented Dec 14, 2022

It is currently not possible to pass through an out argument from FileSequence.asarray to the imread function. I'll add it to the next version. Maybe an option can be added to decode directly to the final memory mapped array if the imread function is known to support it...
Anyway, for most cases it would be better to map the segments in each file of the sequence as a zarr store. This is currently only possible for multi-file OME-TIFF or via generating fsspec reference files. A general solution would require support for zarr sharding...

@cgohlke
Copy link
Owner

cgohlke commented Jan 23, 2023

Should be fixed in version 2023.1.23.

@cgohlke cgohlke closed this as completed Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants