Crash on likely non-compliant server

[lnicola]

I noticed that stac-client collections https://datahub.creodias.eu/stac/ crashes in a pretty obscure way (a NoneError, more or less).

The server is probably not exactly compliant, but it would be nice to report a more useful error.

[jsignell]

Hmm yeah I can reproduce. There is a more informative error if you use the python API:

from pystac_client import Client

catalog = Client.open("https://datahub.creodias.eu/stac/")
collections = list(catalog.get_all_collections())

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[3], line 1
----> 1 collections = list(catalog.get_all_collections())

File ~/pystac/pystac/catalog.py:432, in Catalog.get_all_collections(self)
    429 def get_all_collections(self) -> Iterable[Collection]:
    430     """Get all collections from this catalog and all subcatalogs. Will traverse
    431     any subcatalogs recursively."""
--> 432     yield from self.get_collections()
    433     for child in self.get_children():
    434         yield from child.get_collections()

File ~/pystac-client/pystac_client/client.py:431, in Client.get_collections(self)
    429     raise APIError("Invalid response from /collections")
    430 for col in page["collections"]:
--> 431     collection = CollectionClient.from_dict(
    432         col, root=self, modifier=self.modifier
    433     )
    434     call_modifier(self.modifier, collection)
    435     yield collection

File ~/pystac-client/pystac_client/collection_client.py:84, in CollectionClient.from_dict(cls, d, href, root, migrate, preserve_dict, modifier)
     74 @classmethod
     75 def from_dict(
     76     cls,
   (...)
     82     modifier: Optional[Callable[[Modifiable], None]] = None,
     83 ) -> "CollectionClient":
---> 84     result = super().from_dict(d, href, root, migrate, preserve_dict)
     85     # error: Cannot assign to a method  [assignment]
     86     # https://github.com/python/mypy/issues/2427
     87     setattr(result, "modifier", modifier)

File ~/pystac/pystac/collection.py:637, in Collection.from_dict(cls, d, href, root, migrate, preserve_dict)
    635 description = d.pop("description")
    636 license = d.pop("license")
--> 637 extent = Extent.from_dict(d.pop("extent"))
    638 title = d.pop("title", None)
    639 stac_extensions = d.pop("stac_extensions", None)

File ~/pystac/pystac/collection.py:352, in Extent.from_dict(d)
    344 @staticmethod
    345 def from_dict(d: Dict[str, Any]) -> Extent:
    346     """Constructs an Extent from a dict.
    347 
    348     Returns:
    349         Extent: The Extent deserialized from the JSON dict.
    350     """
    351     return Extent(
--> 352         spatial=SpatialExtent.from_dict(d["spatial"]),
    353         temporal=TemporalExtent.from_dict(d["temporal"]),
    354         extra_fields={
    355             k: v for k, v in d.items() if k not in {"spatial", "temporal"}
    356         },
    357     )

File ~/pystac/pystac/collection.py:120, in SpatialExtent.from_dict(d)
    112 @staticmethod
    113 def from_dict(d: Dict[str, Any]) -> SpatialExtent:
    114     """Constructs a SpatialExtent from a dict.
    115 
    116     Returns:
    117         SpatialExtent: The SpatialExtent deserialized from the JSON dict.
    118     """
    119     return SpatialExtent(
--> 120         bboxes=d["bbox"], extra_fields={k: v for k, v in d.items() if k != "bbox"}
    121     )

TypeError: 'NoneType' object is not subscriptable

I will look into how pystac-client can pass that up better.

[jsignell]

If you check the collections at https://datahub.creodias.eu/stac/collections you will see that the spatial extents are indeed set to null.

[lnicola]

That seems right. According to https://github.com/radiantearth/stac-spec/blob/master/collection-spec/collection-spec.md, the extent is required, so I guess you won't be supporting this implementation.

[jsignell]

Right. I think the most pystac-client will do is try to provide a more informative error and rely on the STAC provider to provide a compliant STAC catalog.

[jsignell]

I just transferred this issue to the pystac-client repo (which provides the stac-client CLI) since it's this library that is not raising an informative enough error.