Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload: Retry on exceptions instead of crashing #597

Open
imerr opened this issue Jun 25, 2023 · 2 comments
Open

Upload: Retry on exceptions instead of crashing #597

imerr opened this issue Jun 25, 2023 · 2 comments

Comments

@imerr
Copy link

imerr commented Jun 25, 2023

Hi
I've been trying to upload some things to IA, unfortunately I would come back to the ia upload command having crashed due to connection timeouts very often
It'd be good if those would be handled more gracefully and just retried unless there's a reason they are not currently?
I have patched this in for myself, but that's more of a duct tape fix to get things working (infinite retries with 1s sleep inbetween on exception) and I figured I'd reach out for feedback how this should be handled properly

I see there is already a retry flag, but that only applies to 503 responses. I would suggest to expand that to also include connection errors and the like? Although personally I'd like it to just retry indefinitely on network errors (with proper backoff between attempts?), so maybe expanding the flag with a special value to do that would be good (but that's an issue for another time)

Let me know what the preferred way of handling this is and I'm happy to get that coded out and submit a pr (or if someone else wants to that's fine too)

@maxz
Copy link
Contributor

maxz commented Jun 27, 2023

Exponential backoff. Also it should probably only retry if the server responded with a "100 Continue" for the same action before.

@JustAnotherArchivist
Copy link
Contributor

I don't think 100s are relevant; requests doesn't support Expect: 100-continue anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants