Generalize the titiler_endpoint parameter

[giswqs]

Currently, the default titiler_endpoint is https://api.cogeo.xyz/. It would be useful to generalize the titiler_endpointso that users can provide a custom endpoint. @TomAugspurger

For example, Microsoft Planetary Computer has its own titiler endpoint at https://planetarycomputer.microsoft.com/api/data/v1/docs#/. Ideally, the add_cog_layer() function should be able to automatically detect if the notebook is running within Planetary Computer (e.g., import planetary-computer is successful) and use the custom endpoint.

https://leafmap.org/leafmap/#leafmap.leafmap.Map.add_cog_layer
https://leafmap.org/leafmap/#leafmap.leafmap.Map.add_cog_mosaic

https://planetarycomputer.microsoft.com/api/data/v1

[TomAugspurger]

Thanks for opening this!

So the primary motivation is to avoid sending large amounts of data from Azure blob storage to the devseed tititler endpoint at api.cogeo.xyz. Since the Planetary Computer has a tititler endpoint running in the same data center, it'll be faster to go from Azure Blob Storage to our tiler.

We can't just set titiler_endpoint="https://planetarycomputer.microsoft.com/api/data/v1" since the endpoints don't match up with the default TiTiler application. Methods like https://github.com/giswqs/leafmap/blob/8a5da2b5494fd8814a7e86f36de99bcfbc93c425/leafmap/common.py#L858-L860 and https://github.com/giswqs/leafmap/blob/8a5da2b5494fd8814a7e86f36de99bcfbc93c425/leafmap/common.py#L1077-L1079 make requests to endpoints like f"{titiler_endpoint}/stac/{TileMatrixSetId}/tilejson.json, but the Planetary Computer instead uses /item/ and /collection.

For this to work, we would need some way for leafmap to know which endpoint to call when it needs to add a COG layer, STAC item layer, mosaic, etc. I see two potential options:

1. Encode this logic in leafmap. Have a class TitilerEndpoint class that maps from desired method to endpoint;

class TitilerEndpoint:
    def url_for_stac_item(...): ...
    def url_for_cog(...): ...
    
class PlanetaryComputerEndpoint(TititlerEndpoint):
    def url_for_stac_item(...): ...
    def url_for_cog(...): ...

And then users would provide something like titiler_endpoint=PlanetaryComputerEndpoint(), and maybe allow aliases like `titiler_endpoint="planetary-computer". It might be nice to have an environment variable that controls the default.

2. Define this logic as an endpoint in titiler. This is kind of making a "spec" for TiTiler, a way for a specific deployment to indicate which endpoints are available and where they're available at.

For now, I'd recommend putting this logic in leafmap or some third-party library.

cc @vincentsarago and @mmcfarland in case you're interested.

[giswqs]

@TomAugspurger Good suggestion. I will implement it using option 1. So the three endpoints (cog, collection, mosaic) should look like this?

titiler_endpoint="https://planetarycomputer.microsoft.com/api/data/v1"

 r = requests.get( 
     f"{titiler_endpoint}/item/{TileMatrixSetId}/tilejson.json", params=params 
 ).json() 

 r = requests.get( 
     f"{titiler_endpoint}/collection/{TileMatrixSetId}/tilejson.json", params=params 
 ).json() 

 r = requests.get( 
     f"{titiler_endpoint}/mosaic/{searchid}/{TileMatrixSetId}/tilejson.json", params=params 
 ).json() 

[vincentsarago]

@giswqs FYI we have a new home for TiTIler demo endpoint: https://titiler.xyz (*.cogeo.xyz might disappear at one point).

Note: titiler.xyz, is deployed with the latest titiler 0.4.0 version, while the tiler on the planetary computer is still on 0.3.*. There are quite some difference on the query parameter between the two version: developmentseed/titiler#396

[TomAugspurger]

We should have the Planetary Computer deployment updated sometime in mid January. I'm not sure how that'll affect leafmap, so we might need to sync up once we get our staging endpoint transitioned.

One thing I'm not sure about is the /cog/{TileMatrixSetId}/tilejson.json endpoint. I don't see an equivalent for that in the Planetary Computer deployment. I'll have to check on what that's doing.

[giswqs]

@vincentsarago Thanks for the heads up. I will look into it and update leafmap accordingly.

@TomAugspurger I don't think it will affect leafmap that much. I will make it compatible with both 0.3.* and 0.4.0. The option 1 you suggested plus an environment variable should do the trick.

[giswqs]

@TomAugspurger This feature has been implemented. You can now use one line of code (m.add_stac_layer()) to load STAC asset items from Planetary Computer. Check out the notebook example https://leafmap.org/notebooks/37_planetary_computer

cc: @vincentsarago @lossyrob

import leafmap

m = leafmap.Map()

collection = "landsat-8-c2-l2"
items = "LC08_L2SP_047027_20201204_02_T1"

m.add_stac_layer(collection=collection, items=items, assets="SR_B5,SR_B4,SR_B3", name="Color infrared")
m.add_stac_layer(collection=collection, items=items, expression="SR_B5/SR_B4", colormap_name="greens", name="NDVI Green")
m

Some relevant classes and functions:

https://github.com/giswqs/leafmap/blob/16efea65acf9cc6e4e2f7d0c86883b0c3187cbc8/leafmap/common.py#L18-L19
https://github.com/giswqs/leafmap/blob/16efea65acf9cc6e4e2f7d0c86883b0c3187cbc8/leafmap/common.py#L65-L66
https://github.com/giswqs/leafmap/blob/16efea65acf9cc6e4e2f7d0c86883b0c3187cbc8/leafmap/common.py#L1187-L1195
https://github.com/giswqs/leafmap/blob/16efea65acf9cc6e4e2f7d0c86883b0c3187cbc8/leafmap/leafmap.py#L911-L924

[giswqs]

One thing I'm not sure about is the /cog/{TileMatrixSetId}/tilejson.json endpoint. I don't see an equivalent for that in the Planetary Computer deployment. I'll have to check on what that's doing.

@TomAugspurger I am wondering about this too. Does Planetary Computer has the /cog/{TileMatrixSetId}/tilejson.json endpoint? Can't find it at https://planetarycomputer.microsoft.com/api/data/v1/docs#/

[TomAugspurger]

Great, thanks @giswqs! I just gave it a quick try and things look great. Once it's released to PyPI / conda-forge I'll update the Hub to set the environment variable.

---

Does Planetary Computer has the /cog/{TileMatrixSetId}/tilejson.json endpoint?

We don't implement any of the cog endpoints, primarily since they're mostly duplicative of the /item endpoints with an ?asset=<asset> parameter. We don't have plans to implement that endpoint now, but could if there's enough interest from users that isn't somehow met by the /item endpoints.

[giswqs]

@TomAugspurger Sounds good. Will make a new release shortly

[giswqs]

@TomAugspurger leafmap v.0.7.0 is now available on PyPi. It should also be available on conda-forge within an hour.

https://pypi.org/project/leafmap
conda-forge/leafmap-feedstock#26

[giswqs]

@TomAugspurger v0.7.0 is now available on conda-forge. BTW, you don't need to set the environment variable. By default, when the collection argument is passed to the add_stac_layer() function with default titiler_endpoint=None. the function will use the Planetary Computer endpoint.

https://github.com/giswqs/leafmap/blob/ad872ba3286ea7e3e87e53eabdcc8bc5da0f98f5/leafmap/common.py#L1261-L1262

[TomAugspurger]

Great, thanks. I just merged microsoft/planetary-computer-hub#46 updating it for staging and will deploy that to production once our integration tests pass.

[TomAugspurger]

Forgot to follow up noting that this is deployed to production now. Thanks again @giswqs!

Would it be OK if I added a short example using leafmap with the Planetary Computer STAC API and tiler to our examples repository at https://github.com/microsoft/planetarycomputerexamples? This would be duplicative of https://leafmap.org/notebooks/37_planetary_computer/, but the PlanetaryComputerExamples repository is automatically cloned into the user's home directory, so it'd be convenient to have it there. As much as possible, we try to keep the examples focused on the Planetary Computer, and link heavily back to the upstream library's documentation.

[giswqs]

Sure, please feel free to adapt the notebook example in any way you like. You can remove the first part on general STAC not related to Planetary Computer.