Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve S3 error recognition for download (And also in general) #42

Open
dankwart-de opened this issue Aug 15, 2019 · 1 comment
Open
Labels
enhancement New feature or request
Milestone

Comments

@dankwart-de
Copy link
Contributor

I think for the S3 helper, there are two cases:

  1. The application stops without an exit code => Thus all is fine, because the file was downloaded completely.
  2. The application stops with an error! In this case, we need to check, if the amount of read records equals the amount of requested records. If this condition applies, all is good.
@dankwart-de dankwart-de added the enhancement New feature or request label Aug 15, 2019
@dankwart-de dankwart-de added this to the 1.0.0 milestone Aug 15, 2019
@vinjana
Copy link
Member

vinjana commented Aug 15, 2019

In the spirit of defensive programming, file sizes should always get checked (i.e. was the FASTQ read completely from the S3 source). A filesize check is probably cheap but could protect against programming errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants