Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuring RetryStacIO #1353

Closed
drnextgis opened this issue Jun 21, 2024 · 3 comments
Closed

Configuring RetryStacIO #1353

drnextgis opened this issue Jun 21, 2024 · 3 comments

Comments

@drnextgis
Copy link

drnextgis commented Jun 21, 2024

I'm using pystac-client to search for STAC items and have configured the retry behavior as described in the documentation. After finding the items, I download the assets using stac-asset. However, sometimes I'm encountering this error:

Could not read uri ...

This error originates from this section of the pystac code.

It appears that there is a RetryStacIO class, which seems to be independent of the retry behavior configured at the pystac-client level. My question is: How can I configure RetryStacIO in the same manner as it is done for pystac-client?

Thank you for your assistance.

@drnextgis
Copy link
Author

drnextgis commented Jun 21, 2024

Apparently something like this should work:

from pystac.stac_io import RetryStacIO, StacIO
StacIO.set_default(RetryStacIO)

But still not quite clear what is the proper way of re-using an existing Retry object. Is this the correct approach?

from urllib3 import Retry
from pystac.stac_io import RetryStacIO, StacIO


retry = Retry(
    total=5,
    backoff_factor=1,
    status_forcelist=[502, 503, 504],
    allowed_methods=None,
)

class CustomRetryStacIO(RetryStacIO):
    def __init__(
        self,
        headers: dict[str, str] | None = None,
        retry: Retry | None = retry,
    ):
        super().__init__(headers, retry)

StacIO.set_default(CustomRetryStacIO)

Additionally, the retry property is defined both at the class level and within the constructor in RetryStacIO, which is a bit confusing.

@gadomski
Copy link
Member

gadomski commented Jul 1, 2024

Your examples make sense, do they seem to work? StacIO is generally a bit wonky and confusing -- there's some documentation in https://pystac.readthedocs.io/en/stable/concepts.html#i-o-in-pystac that is more-or-less what you've demonstrated above.

Additionally, the retry property is defined both at the class level and within the constructor in RetryStacIO, which is a bit confusing.

I've opened #1358 to address this.

@drnextgis
Copy link
Author

Yes, they appear to be working. Thank you for confirming and for the PR 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants