Failing to download files when using the discography option.

[ghost]

Works fine when downloading albums however I get this error when downloading discographys. Here's the link I'm am using to download. https://newretrowave.bandcamp.com/

[zontargs]

I've also seen this problem. It doesn't only happen when using the discography option, but also if you queue up too many albums, or even one very long album.

Bandcamp throttles you by refusing connections if you attempt to access too many files over too short of a time. The size checks + download attempts can quickly exceed this limit if you're attempting to work with >100 files at a time.

EDIT: just had it happen after doing a size check on 40 files, then downloading about 30 of them.

An option to skip the size estimate and limit the number of parallel connections (3-5 seems sustainable based on experience with batched wget downloads from Bandcamp) for large operations would be appreciated.

[Otiel]

I suggest you try using the exponential back-off algorithm that has already been implemented. Can you try adding these lines to your settings file (BcDlSettings.json)?

"DownloadMaxTries": 7
"DownloadRetryCooldown": 0.2
"DownloadRetryExponential": 4.0

Check the wiki for more info.

It should work as shown below:

Let me know if that helps.

[zontargs]

Thanks. I've enabled that setting, and it was a moderate improvement. Without "download one album at a time" enabled, the download failures tend to clump together, resulting in simultaneous retries tripping the spam detector, continuing the cycle. It appears to be less of an issue when doing one album at a time.

[Otiel]

By "tripping the spam detector", you mean there are several failures, but before the max retry count (7), each song is correctly downloaded right?

[zontargs]

Negative. Because many of the 7th attempts happen nearly simultaneously, a large number of those will fail in a large project. I attempted to download a 284 file album, with 69 files failing to download on the 7th attempt.

Also of note: even an hour after the last download was attempted, the operation had not terminated. The program simply sat there after the last download attempt, without completing and printing a summary:
http://i.thinimg.com/img/cux.png

While the exponential back-off improves speed on smaller projects, an option to limit the number of simultaneous connection attempts would be better for large projects. What works for a sprint does not work well for a marathon.

[Otiel]

Ok, I'll work on adding such an option when I find the time. Thanks for your tests and suggestions.