Retry to download failed batches, Issues #16 & #31 #36
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a new function that retries to download failed batches
iimog
approved these changes
Nov 2, 2021
There was a problem hiding this comment.
Awesome! Thank you for the contribution. It is very desirable to make bcdatabaser more robust, particularly regarding download errors. These are very often temporary and retrying after a minute is a good idea. Also thanks for cleaning up some whitespace errors and fixing the link in the README.
| - File::Path | ||
| - [NCBI::Taxonomy](https://github.com/greatfireball/NCBI-Taxonomy) | ||
|
|
||
| External Programs: | ||
| - [NCBI eutils](https://www.ncbi.nlm.nih.gov/books/NBK25500) | ||
| - [KronaTools](https://github.com/marbl/Krona) | ||
| - [SeqFilter](https://github.com/BioInf-Wuerzburg/SeqFilter) | ||
| - [dispr](https://github.com/molbiodiv/metabDB/issues) | ||
| - [dispr](https://github.com/douglasgscofield/dispr) |
|
Thanks for the merge! The whitespace fixes weren't intentional, so I guess the credit goes to my text editor. I found the problem with the link to dispr while trying a native installation, because i didn't have much experience in working with docker. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If the download of a batch failes, BCdatabaser now tries to download it again after 60 seconds, up to a maximum of three tries. As a result, the tool does not have to be completely rerun in the event of brief connection interruptions, which is particularly helpful when working with large databases.
You can find a Docker image of this branch here.
A logfile from a run with 5 downloading errors and used command (trnl, all viridiplantae) can be found here. You can find the errors by searching for "WARNING".
This my first time contributing to someone elses project and my first time writing perl code, I'm open to suggestions of all kinds.