Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved error-handling in S3 completion callback #504

Merged
merged 1 commit into from
Sep 27, 2023
Merged

Conversation

graebm
Copy link
Contributor

@graebm graebm commented Sep 27, 2023

Issue:
A user experienced random downloads hanging. I looked at their code and saw that an exception in their completion callback could lead to a hang, since later code was never executed. Then I looked at our code and saw a theoretical place where the same kinda thing could happen, an error could prevent us invoking the completion callback.

Description of changes:

  • Always invoke the completion callback, even if we failed to turn the error_response_headers into PyObjects. These headers are optional anyway.
  • Use nice new error code AWS_ERROR_FILE_WRITE_FAILURE if file write fails.
  • Raise error if file close fails, because that probably means it failed to flush the buffers to disk.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@graebm graebm merged commit cf9b73f into main Sep 27, 2023
64 checks passed
@graebm graebm deleted the s3-error-handling branch September 27, 2023 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants