Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

call .read() only once when dataset do not have nodata #355

Merged
merged 2 commits into from Feb 19, 2021

Conversation

vincentsarago
Copy link
Member

@vincentsarago vincentsarago commented Feb 19, 2021

For a dataset that has a mask or and alpha band (by default every data that don't have Nodata set will have an alpha/mask band within the WarpedVRT instance) we were doing two read(), one for the data, another one for the mask. This PR changes the behavior and performs only one read() and then split the data and the mask to two arrays.

### Before 
$ python tiprofile.py https://rio-tiler-dev.s3.amazonaws.com/data/m_3209542_se_15_060_20181115_20190222.tif --tile 13-1916-3319 --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
         193901 function calls (193178 primitive calls) in 0.978 seconds

   Ordered by: internal time, cumulative time
   List reduced from 635 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.571    0.571    0.571    0.571 __init__.py:56(open)
        2    0.118    0.059    0.118    0.059 {method 'read' of 'rasterio._warp.WarpedVRTReaderBase' objects}
 
### Now
$ python tiprofile.py https://rio-tiler-dev.s3.amazonaws.com/data/m_3209542_se_15_060_20181115_20190222.tif --tile 13-1916-3319 --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR       
         193801 function calls (193078 primitive calls) in 0.885 seconds

   Ordered by: internal time, cumulative time
   List reduced from 636 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.382    0.382    0.383    0.383 __init__.py:56(open)
        1    0.048    0.048    0.048    0.048 {method 'read' of 'rasterio._warp.WarpedVRTReaderBase' objects}

☝️ this is a profiling results that show that the method 'read' of 'rasterio._warp.WarpedVRTReaderBase' objects was called twice and only once in the new version.

@vincentsarago vincentsarago merged commit d4c2763 into master Feb 19, 2021
@vincentsarago vincentsarago deleted the onlyOneRead branch February 19, 2021 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant