[FR] Allow resuming in fasterq-dump

[ArchKudo]

I'm using fasterq-dump to download and extract fastq files of all runs (~250 files, 50GB each) of project PRJEB31266
However I find it very flaky, failing multiple times. Retrying it cause to download & extract the entire file from scratch which is wasteful in terms of bandwidth and processing.
It would be nice if it had something similar to --resume flag present in prefetch

[klymenko]

Did you run prefetch before fasterq-dump?

[ArchKudo]

Hello, thank you for replying.
I ended up using prefetch before fasterq-dump.
However, this didn't fix the problem since fasterq-dump was utilizing more space than what was available in tmp/ and required utilizing the --temp flag.

I still find value in implementing a resume feature, as the extraction process takes longer compared to downloading the sra file. Moreover, it might fail midway due to reasons such as running out of tmp/ storage. However, I leave it up to you to decide whether to close this issue.

[kcmtest]

Hello, thank you for replying. I ended up using prefetch before fasterq-dump. However, this didn't fix the problem since fasterq-dump was utilizing more space than what was available in tmp/ and required utilizing the --temp flag.

I still find value in implementing a resume feature, as the extraction process takes longer compared to downloading the sra file. Moreover, it might fail midway due to reasons such as running out of tmp/ storage. However, I leave it up to you to decide whether to close this issue.

how did you fix im also similar situation where i have like more than 1000 files to download but it fails after 300-350 files, even i use prefetch first so in my case the prefetch is not able to get the given ID. Is it a space issue ? " it might fail midway due to reasons such as running out of tmp/ storage." can you explain