Numpy 2.0 breaks opening of certain malformed tiffiles

[NewSoupVi]

I am using the geotiff library (which uses this library for reading the tif files) with this tif:
https://land.copernicus.eu/en/products/high-resolution-layer-tree-cover-density/tree-cover-density-2018

This file reports a type of "byte" (uint8) and has a nodata value of -999999999.

Which is obviously impossible, so this .tif is malformed.

Thing is, it still opened just fine previously and I was able to work with it just fine.
However, with yesterday's release of Numpy 2.0, this tiffile now no longer opens, because it crashes inside the zarr library on this line:
https://github.com/zarr-developers/zarr-python/blob/v3/src/zarr/v2/meta.py#L250
Numpy now makes sure that the dtype and the value are actually compatible. (which is probably a good thing)

I am not sure whether this is your problem, or theirs, or just a problem with the .tif file.
Either way, it is quite annoying, and it would be nice if there was maybe a way to override the nodata value, or some sort of special handling that makes sure the dtype and the nodata value are compatible, and if not, allows some way around that.

I have opened an equivalent issue on the zarr repository: zarr-developers/zarr-python#1981

Thanks for your time!

[cgohlke]

Please provide a file to reproduce this issue.

[NewSoupVi]

Oof, I guess I'll have to make one myself? Don't really know how to do that, but I guess I can try. I did link one but yeah, it is annoying to get access to

[NewSoupVi]

Ok, I got it.

https://drive.google.com/file/d/1gMDrYWQ4SgA9YUxiRcNmDMuY2F79_51k/view?usp=sharing

With this file (when having numpy 2.0.0 installed, and not when having numpy 1.#.# installed), if I run this:

store = tifffile.imread("test.tiff", aszarr=True)
z = zarr.open(store, mode='r')

I get this exception:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\meta.py", line 127, in decode_array_metadata
    fill_value = cls.decode_fill_value(meta["fill_value"], dtype, object_codec)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\meta.py", line 260, in decode_fill_value
    return np.array(v, dtype=dtype)[()]
           ^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -99999999 out of bounds for uint8

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\convenience.py", line 133, in open
    return open_array(_store, mode=mode, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\creation.py", line 689, in open_array
    z = Array(
        ^^^^^^
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\core.py", line 170, in __init__
    self._load_metadata()
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\core.py", line 193, in _load_metadata
    self._load_metadata_nosync()
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\core.py", line 207, in _load_metadata_nosync
    meta = self._store._metadata_class.decode_array_metadata(meta_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\zarr\meta.py", line 141, in decode_array_metadata
    raise MetadataError("error decoding metadata") from e
zarr.errors.MetadataError: error decoding metadata

[cgohlke]

Thanks. The next version of tifffile verifies that the value of the GDAL_NODATA tag is compatible with the data type and logs a warning if not.

[cgohlke]

Fixed in v2024.6.18.

[NewSoupVi]

Thank you so much!! :)