Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: GTiff: Allow concurrent reading of single blocks #8448

Open
lnicola opened this issue Sep 21, 2023 · 2 comments
Open

FR: GTiff: Allow concurrent reading of single blocks #8448

lnicola opened this issue Sep 21, 2023 · 2 comments

Comments

@lnicola
Copy link
Contributor

lnicola commented Sep 21, 2023

GTiff supports multi-threaded reads by using PRead (and, I suspect, by doing the I/O outside of libtiff/libgeotiff). It would be useful if this could be extended to reading single blocks. Currently, the alternatives are:

  • not using threads
  • opening the file multiple times: this works, but is not really cache-efficient, and doesn't work well with the low default NOFILE limit
  • reading larger regions using the existing threading support: this will be less efficient when the caller knows it only needs to touch a limited number of blocks
@lnicola lnicola changed the title FR: Allow concurrent reading of single blocks FR: GTiff: Allow concurrent reading of single blocks Sep 21, 2023
@rouault
Copy link
Member

rouault commented Sep 21, 2023

if this could be extended to reading single blocks

I don't see any easy resolution for that. This goes against a lot of assumptions in the design of GDAL. If that was going to be implemented, it would have implications probably on core and should be done driver by driver, with a lot of caution to lock the appropriate data structures, and probably a driver capability advertising that sch multi-threaded reads can be safely done. For GTiff, not only the file access should be done in a per-thread way, but a TIFF* handle also hides a lot of state and shouldn't be used from several threads.

@latot
Copy link

latot commented Sep 21, 2023

This seems great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants