large_image tiling issue with specific dataset

[banesullivan]

Recently, large_image started failing to serve tiles for this dataset: https://data.kitware.com/#item/5f9c316650a41e3d1920ef90

We were originally using this raster for some testing of the large_image tile serving endpoint, so I know it worked at one point.

Recently, I started seeing this numpy error when trying to serve tiles for this file:

...
                                  File "/usr/local/lib/python3.8/site-packages/nu
django_1    |                     mpy/core/getlimits.py", line 523, in __init__
django_1    |                         raise ValueError("Invalid integer data type
django_1    |                     %r." % (self.kind,))
django_1    |                     ValueError: Invalid integer data type 'f'.

django_1    | [16:42:08] ERROR    Internal Server Error: /api/geodata/imagery/image log.py:224
django_1    |                     _entry/159/tiles/12/830/1565.png
django_1    |                     Traceback (most recent call last):
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/exception.py", line 47, in
django_1    |                     inner
django_1    |                         response = get_response(request)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/base.py", line 181, in
django_1    |                     _get_response
django_1    |                         response = wrapped_callback(request,
django_1    |                     *callback_args, **callback_kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/views/decorators/csrf.py", line 54, in
django_1    |                     wrapped_view
django_1    |                         return view_func(*args, **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/views/generic/base.py", line 70, in view
django_1    |                         return self.dispatch(request, *args,
django_1    |                     **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 509, in dispatch
django_1    |                         response = self.handle_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 469, in
django_1    |                     handle_exception
django_1    |                         self.raise_uncaught_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 480, in
django_1    |                     raise_uncaught_exception
django_1    |                         raise exc
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 506, in dispatch
django_1    |                         response = handler(request, *args, **kwargs)
django_1    |                       File
django_1    |                     "/opt/django-project/rgd/geodata/api/tiles.py",
django_1    |                     line 36, in get
django_1    |                         tile_binary = tile_source.getTile(x, y, z)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/cache_util/cache.py", line 102, in
django_1    |                     wrapper
django_1    |                         v = func(self, *args, **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image_source_gdal/__init__.py", line 692, in
django_1    |                     getTile
django_1    |                         return self._outputTile(tile,
django_1    |                     TILE_FORMAT_NUMPY, x, y, z,
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1486, in
django_1    |                     _outputTile
django_1    |                         tile, mode = self._outputTileNumpyStyle(tile,
django_1    |                     applyStyle, x, y, z, kwargs.get('frame'))
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1444, in
django_1    |                     _outputTileNumpyStyle
django_1    |                         tile = self._applyStyle(tile, self.style, x,
django_1    |                     y, z, frame)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1406, in
django_1    |                     _applyStyle
django_1    |                         min = self._getMinMax('min', entry.get('min',
django_1    |                     'auto'), image.dtype, bandidx, frame)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1318, in
django_1    |                     _getMinMax
django_1    |                         self._scanForMinMax(dtype, frame)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image_source_gdal/__init__.py", line 264, in
django_1    |                     _scanForMinMax
django_1    |                         self._bandRanges[frame]['max'], numpy.iinfo(s
django_1    |                     elf._bandRanges[frame]['max'].dtype).max)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/nu
django_1    |                     mpy/core/getlimits.py", line 523, in __init__
django_1    |                         raise ValueError("Invalid integer data type
django_1    |                     %r." % (self.kind,))
django_1    |                     ValueError: Invalid integer data type 'f'.
django_1    |            ERROR    Internal Server Error: /api/geodata/imagery/image log.py:224
django_1    |                     _entry/159/tiles/12/831/1566.png
django_1    |                     Traceback (most recent call last):
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/exception.py", line 47, in
django_1    |                     inner
django_1    |                         response = get_response(request)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/base.py", line 181, in
django_1    |                     _get_response
django_1    |                         response = wrapped_callback(request,
django_1    |                     *callback_args, **callback_kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/views/decorators/csrf.py", line 54, in
django_1    |                     wrapped_view
django_1    |                         return view_func(*args, **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/views/generic/base.py", line 70, in view
django_1    |                         return self.dispatch(request, *args,
django_1    |                     **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 509, in dispatch
django_1    |                         response = self.handle_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 469, in
django_1    |                     handle_exception
django_1    |                         self.raise_uncaught_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 480, in
django_1    |                     raise_uncaught_exception
django_1    |                         raise exc
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 506, in dispatch
django_1    |                         response = handler(request, *args, **kwargs)
django_1    |                       File
django_1    |                     "/opt/django-project/rgd/geodata/api/tiles.py",
django_1    |                     line 36, in get
django_1    |                         tile_binary = tile_source.getTile(x, y, z)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/cache_util/cache.py", line 102, in
django_1    |                     wrapper
django_1    |                         v = func(self, *args, **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image_source_gdal/__init__.py", line 692, in
django_1    |                     getTile
django_1    |                         return self._outputTile(tile,
django_1    |                     TILE_FORMAT_NUMPY, x, y, z,
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1486, in
django_1    |                     _outputTile
django_1    |                         tile, mode = self._outputTileNumpyStyle(tile,
django_1    |                     applyStyle, x, y, z, kwargs.get('frame'))
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1444, in
django_1    |                     _outputTileNumpyStyle
django_1    |                         tile = self._applyStyle(tile, self.style, x,
django_1    |                     y, z, frame)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1407, in
django_1    |                     _applyStyle
django_1    |                         max = self._getMinMax('max', entry.get('max',
django_1    |                     'auto'), image.dtype, bandidx, frame)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/la
django_1    |                     rge_image/tilesource/base.py", line 1335, in
django_1    |                     _getMinMax
django_1    |                         value =
django_1    |                     self._bandRanges[frame]['max'][bandidx]
django_1    |                     IndexError: index 1 is out of bounds for axis 0
django_1    |                     with size 1

@manthey, any insight here?

FYI, this raster is a part of our demo_data command.

[manthey]

I get the same error even on the large image version from last July (v1.2.0). I didn't check further back. I'll look into what the issue is. Addendum: This also isn't related to the GDAL version. It has the same error with GDAL 3.1.0 and latest master.

[manthey]

The file can be read with the tiff tile source (not geospatially).

[manthey]

There is fixed by girder/large_image#565.

[banesullivan]

Great! Thanks, @manthey! Do you plan on tagging a release for this fix?

[manthey]

Yes, I'll do so shortly.

[manthey]

A new version available (1.4.3).

[banesullivan]

I can confirm all is working now. I'm not going to set large_image>=1.4.3 in setup.py and we'll just assume the latest version gets pulled in