Are data with "PIXELTYPE=SIGNEDBYTE" supported? #2926

nicolasvila · 2023-09-25T08:48:55Z

nicolasvila
Sep 25, 2023

Hi,
I'm not sure whether this is a RasterIO or GDAL issue but I'm facing cascading issues with RasterIO (with rio_tiler plugin) and Titiler while handling images that have this particular pixel type. Seems that the image bands are considered to have different pixels' types and returns a list. According to the metadata this is obviously not the case as we can see in the example below.

## Here's the error message I got

2023-09-18T14:53:52.076875911Z   File "/usr/local/lib/python3.9/site-packages/rio_tiler/io/base.py", line 118, in tile_exists
2023-09-18T14:53:52.076877954Z     (tile_bounds[0] < dst_bounds[2])
2023-09-18T14:53:52.076880074Z TypeError: '<' not supported between instances of 'float' and 'list'

I apologize if this has to be taken into account at the higher (application) level, in that case it would be on rio_tiler or Titiler side which do not expect to receive a list of pixel types (one for each band whereas they are all the same)
Possible linked issue: #2180 and this PR: #1595
This also may be related to this GDAL issue: OSGeo/gdal#2298

GDAL info output on the test COG file

gdalinfo IMG_COG.TIFF|grep -A3 -i structure

Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=PIXEL
  LAYOUT=COG
--
  Image Structure Metadata:
    PIXELTYPE=SIGNEDBYTE
Band 2 Block=512x512 Type=Byte, ColorInterp=Green
  Overviews: 915x915, 458x458, 229x229
  Image Structure Metadata:
    PIXELTYPE=SIGNEDBYTE
Band 3 Block=512x512 Type=Byte, ColorInterp=Blue
  Overviews: 915x915, 458x458, 229x229
  Image Structure Metadata:
    PIXELTYPE=SIGNEDBYTE

Operating system

Ubuntu 22.04 64 bit

Software versions

rasterio-1.3.8
rio_tiler-4.1.13
GDAL 3.6.4, released 2023/04/17

vincentsarago · 2023-09-25T09:08:09Z

vincentsarago
Sep 25, 2023
Maintainer

@nicolasvila Can you share some reproductible example (simple rio-tiler read block), and if possible a data sample?

if you're note confortable sharing your dataset you can either use https://github.com/cogeotiff/rio-faux or send it directly to me vincent@developmentseed.org 🙏

The error message is kinda weird because I don't see how the bounds could be a list and a float.

0 replies

nicolasvila · 2023-09-26T08:16:23Z

nicolasvila
Sep 26, 2023
Author

Hi Vincent,
Thanks for taking a look at this. I put a test file here if you want to play with it and check for yourself:
https://github.com/nicolasvila/test-data/tree/main/s250-advanced
I believe it's an error related to GDAL but rio_tiler and titler could probably mitigate the issue and/or give a better error message.
I haven't tested with version 3.7 of GDAL, which seems to have important code refactor. Translating data to "BYTE" fix the issue so titler can handle it.

1 reply

vincentsarago Sep 26, 2023
Maintainer

Thanks @nicolasvila, I can't reproduce the issue locally 👇

from rio_tiler.io import Reader
from rio_tiler import __version__

print(__version__)

>> 6.1.0

with Reader("IMG_MSI_TCI_R60m_S2A_MSI__SGP_20220930T161339_af98.COG.TIFF") as src:
    img = src.preview()
    print(src.bounds)
    print(src.geographic_bounds)
    print(src.info())
    tile = src.tile(*src.tms.tile(-80, 28, src.minzoom))
    print(tile)

>> (499980.0, 3090240.0, 609780.0, 3200040.0)
(-81.0002051974698, 27.932627674826595, -79.87376087948627, 28.928282733242952)
bounds=BoundingBox(left=-81.0002051974698, bottom=27.932627674826595, right=-79.87376087948627, top=28.928282733242952) minzoom=8 maxzoom=11 band_metadata=[('b1', {}), ('b2', {}), ('b3', {})] band_descriptions=[('b1', ''), ('b2', ''), ('b3', '')] dtype='int8' nodata_type='None' colorinterp=['red', 'green', 'blue'] scale=None offset=None colormap=None driver='GTiff' count=3 width=1830 height=1830 overviews=[2, 4, 8]
ImageData(array=masked_array(
  data=[[[0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         ...,
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --]],

        [[0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         ...,
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --]],

        [[0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         [0, 0, 0, ..., --, --, --],
         ...,
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --],
         [--, --, --, ..., --, --, --]]],
  mask=[[[False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         ...,
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True]],

        [[False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         ...,
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True]],

        [[False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         [False, False, False, ...,  True,  True,  True],
         ...,
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True],
         [ True,  True,  True, ...,  True,  True,  True]]],
  fill_value=999999,
  dtype=int8), assets=['IMG_MSI_TCI_R60m_S2A_MSI__SGP_20220930T161339_af98.COG.TIFF'], bounds=BoundingBox(left=-8922952.93389838, bottom=3130860.6785608903, right=-8766409.89997034, top=3287403.712488929), crs=CRS.from_epsg(3857), metadata={}, band_names=['b1', 'b2', 'b3'], dataset_statistics=None, cutline_mask=None)

 rio --show-versions
rasterio info:
  rasterio: 1.3.8
      GDAL: 3.6.4
      PROJ: 9.0.1
      GEOS: 0.0.0
 PROJ DATA: /opt/homebrew/share/proj
 GDAL DATA: /opt/homebrew/share/gdal

System:
    python: 3.9.17 (main, Jun 20 2023, 18:00:22)  [Clang 14.0.3 (clang-1403.0.22.14.1)]
executable: /Users/vincentsarago/Dev/venv/py39/bin/python
   machine: macOS-13.4-arm64-arm-64bit

Python deps:
    affine: 2.4.0
     attrs: 23.1.0
   certifi: 2023.07.22
     click: 8.1.6
     cligj: 0.7.2
    cython: None
     numpy: 1.23.5
    snuggs: 1.4.7
click-plugins: None
setuptools: 68.0.0

nicolasvila · 2023-09-26T09:49:17Z

nicolasvila
Sep 26, 2023
Author

Strange. I'm trying to stream it from a S3 storage. You can try from this link
I also attached a log file.

My versions:

rasterio info:
  rasterio: 1.3.4
      GDAL: 3.6.4
      PROJ: 9.1.1
      GEOS: 3.11.1
 PROJ DATA: /home/nvila/.local/share/proj:/usr/share/proj
 GDAL DATA: None

System:
    python: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
executable: /usr/bin/python3
   machine: Linux-6.2.0-32-generic-x86_64-with-glibc2.35

Python deps:
    affine: 2.3.0
     attrs: 23.1.0
   certifi: 2020.06.20
     click: 8.1.3
     cligj: 0.7.2
    cython: 0.29.28
     numpy: 1.21.5
    snuggs: 1.4.7
click-plugins: None
[log file.txt](https://github.com/rasterio/rasterio/files/12725955/log.file.txt)

setuptools: 59.6.0

log file.txt

8 replies

nicolasvila Sep 28, 2023
Author

Yes, it impacts titiler but also Planetary Computer APIs, as the code might be outdated as well
https://github.com/microsoft/planetary-computer-apis

vincentsarago Sep 28, 2023
Maintainer

Yes the planetary computer is still on rio-tiler 4.0

nicolasvila Sep 29, 2023
Author

Any plan to upgrade pctiler in the pipe? I didn't look closely into the code to check the impacts

vincentsarago Sep 29, 2023
Maintainer

Not that I know of, you may open an issue in MS repo and see if they are able to update the tiler!

nicolasvila Oct 2, 2023
Author

I will raise an issue on the MS repo.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Are data with "PIXELTYPE=SIGNEDBYTE" supported? #2926

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 9 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Are data with "PIXELTYPE=SIGNEDBYTE" supported? #2926

nicolasvila Sep 25, 2023

GDAL info output on the test COG file

Operating system

Software versions

Replies: 3 comments · 9 replies

vincentsarago Sep 25, 2023 Maintainer

nicolasvila Sep 26, 2023 Author

vincentsarago Sep 26, 2023 Maintainer

nicolasvila Sep 26, 2023 Author

nicolasvila Sep 28, 2023 Author

vincentsarago Sep 28, 2023 Maintainer

nicolasvila Sep 29, 2023 Author

vincentsarago Sep 29, 2023 Maintainer

nicolasvila Oct 2, 2023 Author

nicolasvila
Sep 25, 2023

Replies: 3 comments 9 replies

vincentsarago
Sep 25, 2023
Maintainer

nicolasvila
Sep 26, 2023
Author

vincentsarago Sep 26, 2023
Maintainer

nicolasvila
Sep 26, 2023
Author

nicolasvila Sep 28, 2023
Author

vincentsarago Sep 28, 2023
Maintainer

nicolasvila Sep 29, 2023
Author

vincentsarago Sep 29, 2023
Maintainer

nicolasvila Oct 2, 2023
Author