Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API error when trying to download various datasets from planetary computer APIs #176

Open
pv-98 opened this issue May 19, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@pv-98
Copy link

pv-98 commented May 19, 2023

I am trying to download the sentinel-2 dataset via STAC API, but midway through the download I am getting an error like this

APIError: <!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' '[http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html](http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'%3E%3Chtml) xmlns='[http://www.w3.org/1999/xhtml'><head><meta](http://www.w3.org/1999/xhtml'%3E%3Chead%3E%3Cmeta) content='text/html; charset=utf-8' http-equiv='content-type'/><style type='text/css'>body {font-family:Arial; margin-left:40px; }img { border:0 none; }#content { margin-left: auto; margin-right: auto }#message h2 { font-size: 20px; font-weight: normal; color: #000000; margin: 34px 0px 0px 0px }#message p { font-size: 13px; color: #000000; margin: 7px 0px 0px0px}#errorref { font-size: 11px; color: #737373; margin-top: 41px }</style><title>Service unavailable</title></head><body><div id='content'><div id='message'><h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p></div><div id='errorref'><span>0CURnZAAAAACaXluYute5QouGLuN7eMExQlJVMzBFREdFMTExNgA5MjdhYmZhNi0xOWY2LTRhZjEtYTA5ZC1jOTU5ZDlhMWU2NDQ=</span></div></div></body></html>

I have downloaded the dataset the previous day without any issue, but I am not able to do it now. And the error appears at random places midway through the download. This error appears when trying to download many such datasets like JRC-GSW, TerraClimate, COP-DEM-GLO-30 and ESRI-10 meter Land cover.

Can someone please help me with this error.

Any help to resolve this issue would be greatly appreciated.

Thank You.

@pv-98 pv-98 added the bug Something isn't working label May 19, 2023
@TomAugspurger
Copy link
Contributor

In general, we recommend you not download the data locally. See https://tomaugspurger.github.io/noaa-nwm/02-problems.html#background.

but midway through the download I am getting an error like this

What URL is being accessed when that fails? Can you show the full traceback / error message?

@pv-98
Copy link
Author

pv-98 commented May 19, 2023

image image image

Thanks for the reply, here is the full traceback

@TomAugspurger
Copy link
Contributor

So that indicates the error happened on a request to our STAC API, made by pystac / pystac-client.

In general, you should ensure that your network requests includes retries. I'm not sure about the best way to achieve that through pystac-client. I see that stac-utils/pystac#986 added retry logic to pystac; I'm not sure if that's usable through pystac-client.

I'd recommend opening a new issue on pystac-client, requesting that retry configuration be documented (and perhaps enabled by default).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants