Use pystac_client behind proxy server

[lukasValentin]

We are using pystac_client for searching satellite data. Unfortunately, we are located behind a proxy server. Therefore, we get an APIError when we try to connect to Microsoft Planetary Computer STAC:

cat = Client.open(url='https://planetarycomputer.microsoft.com/api/stac/v1')

throws

APIError: HTTPSConnectionPool(host='planetarycomputer.microsoft.com', port=443): Max retries exceeded with url: /api/stac/v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)')))

We know that we can bypass the proxy server using urllib3:

import os
from urllib3 import ProxyManager

proxies = {
      "http"  : os.environ.get('http_proxy'), 
      "https" : os.environ.get('https_proxy')
 }
proxy_manager = ProxyManager(proxies['https'])

We also saw that pystac_client's Client class since recently accepts the keyword request_modifier (0390adf). This keyword expects a Callable that returns a Request object.

However, we did not find a working solution to write such a Callable allowing us to include the ProxyManager to make the requests from pystac_client work.

Is there a solution at hand or is it planned to support proxy-configurations by default in pystac_client? This would help us a lot!

Cheers,
Lukas

[gadomski]

pystac-client's io is pretty tightly integrated with requests. Theoretically, the parameter-transformation operations could be separated from the request-building operations, which would make it easier to implement custom IO libraries. But that doesn't exist right now -- your best bet with the current codebase would be to implement your own StacIO based on urllib3.

I'm tagging this as an enhancement, because it would be nice to decouple the request-building from the actual requests library.

[benz0li]

According to the error message, the HTTP proxy itself is not the problem, but rather the Root Certificate that is used there.

@lukasValentin Please try Advanced Usage - urllib3 2.0.0a2 documentation > Custom TLS Certificates¹ and tell me if that works.

@gadomski When using requests, things are easy: Advanced Usage — Requests 2.28.1 documentation > SSL Cert Verification.

Footnotes

1. Certifi does not support any addition/removal or other modification of the CA trust store content. ↩

[gadomski]

If you just need to use a custom certificate, then you should be able to set verify on the StacApiIO.session attribute, e.g. (untested)

from pystac_client import StacApiIO, Client
stac_api_io = StacApiIO()
stac_api_io.session.verify = "/path/to/certfile"
client = Client.from_file("https://planetarycomputer.microsoft.com/api/stac/v1", stac_io=stac_api_io)

@lukasValentin let me know if you're able to find a fix from my or @benz0li's suggestions.

[lukasValentin]

Thanks a lot @benz0li and @gadomski for your help. Indeed, @gadomski 's suggestions worked out and I was able to bypass the proxy server issue by specifying our custom certificate. I only had to make a minor change to the import statements (I'm using pystac_client pystac-client==0.5.1):

from pystac_client.stac_api_io import StacApiIO
from pystac_client.client import Client

From my side this issue is resolved.