Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add retry coverage to the streaming portion of a download #241

Merged
merged 7 commits into from
Jul 27, 2021

Conversation

cojenco
Copy link
Contributor

@cojenco cojenco commented Jul 20, 2021

This broadens the retry coverage in downloads to cover both the initial response (receiving/evaluating http headers) and the streaming/writing to file portion of the download.

  • Modifies the original consume() method to an internal method _consume() and replace the retry-able http request to a raw http request
  • Adds an internal method consume_with_retries() within classes Download and ChunkedDownload that will attempt to retry a callable _consume() until success
  • Updates tests and docstrings

Fixes #240

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/google-resumable-media-python API. label Jul 20, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 20, 2021
@cojenco cojenco requested a review from andrewsg July 20, 2021 19:36
@cojenco cojenco marked this pull request as ready for review July 27, 2021 23:12
@cojenco cojenco requested a review from a team as a code owner July 27, 2021 23:12
Copy link
Contributor

@andrewsg andrewsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this awesome work!

@andrewsg andrewsg merged commit cc1f07c into googleapis:master Jul 27, 2021
cojenco added a commit to cojenco/google-resumable-media-python that referenced this pull request Jul 30, 2021
cojenco added a commit to cojenco/google-resumable-media-python that referenced this pull request Jul 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/google-resumable-media-python API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exception retries do not cover mid-stream errors which occur during downloads.
2 participants