Control sleep between downloads

[bornio]

I would love to be able to manually control, via a flag, the delay between downloads, instead of the hardcoded value in e.g. https://github.com/rg3/youtube-dl/blob/244cd04237fe4a1e4d92421711f41de3c2566d5c/youtube_dl/extractor/pluralsight.py#L214:

                # Pluralsight tracks multiple sequential calls to ViewClip API and start
                # to return 429 HTTP errors after some time (see
                # https://github.com/rg3/youtube-dl/pull/6989). Moreover it may even lead
                # to account ban (see https://github.com/rg3/youtube-dl/issues/6842).
                # To somewhat reduce the probability of these consequences
                # we will sleep random amount of time before each call to ViewClip.
                self._sleep(
                    random.randint(2, 5), display_id,
                    '%(video_id)s: Waiting for %(timeout)s seconds to avoid throttling')

[dstftw]

Delay between downloads is controlled by --sleep-interval. Ability to control this interval will result in almost immediate bans when it's disabled.

[bornio]

@dstftw That doesn't seem to impact playlists. Once in a playlist download, only the internal sleep of randint(2,5) is in effect.

[dstftw]

It does:

> youtube-dl --sleep-interval 10 https://www.pluralsight.com/courses/angularjs-get-started
[pluralsight:course] angularjs-get-started: Downloading course JSON
[pluralsight:course] angularjs-get-started: Downloading course data JSON
[download] Downloading playlist: AngularJS: Get Started
[pluralsight:course] playlist AngularJS: Get Started: Collected 44 video ids (downloading 44 of them)
[download] Downloading video 1 of 44
[pluralsight] angularjs-get-started-m1-introduction-0: Downloading webpage
[pluralsight] angularjs-get-started-m1-introduction-0: Downloading mp4-high URL
[pluralsight] angularjs-get-started-m1-introduction-0: Waiting for 4 seconds to avoid throttling
[download] Sleeping 10.0 seconds...
[download] Destination: An Introduction - Hello!-angularjs-get-started-m1-introduction-01-fmp4-high.mp4
[download] 100% of 1.21MiB in 00:00
[download] Downloading video 2 of 44
...
[download] Sleeping 10.0 seconds...
[download] Destination: An Introduction - AngularJS-angularjs-get-started-m1-introduction-02-fmp4-high.mp4
[download] 100% of 10.00MiB in 00:06
[download] Downloading video 3 of 44
...
[download] Sleeping 10.0 seconds...
...

[durai23]

Delay between downloads is controlled by --sleep-interval. Ability to control this interval will result in almost immediate bans when it's disabled.

I didn't understand the exact meaning of what you said. Do you mean that if you do not use this option (and do not use proxies) your IP will get banned? Also, what is your suggested sleep interval?

[DeveloperMCD]

Why does the --sleep-interval not kick in when a file "has already been downloaded and merged"?

[download] Downloading video 1 of 905
[youtube] zVqbC3C_2Uo: Downloading webpage
[youtube] zVqbC3C_2Uo: Downloading video info webpage
[download] Sleeping 90 seconds...
Destination: Key group behind Amazon Synod is funded by large U.S. pro-abortion charity-zVqbC3C_2Uo.f136.mp4
[download] 100% of 21.57MiB in 00:00
[download] Sleeping 90 seconds...
[download] Destination: Key group behind Amazon Synod is funded by large U.S. pro-abortion charity-zVqbC3C_2Uo.f140.m4a
[download] 100% of 2.72MiB in 00:00
[ffmpeg] Merging formats into "Key group behind Amazon Synod is funded by large U.S. pro-abortion charity-zVqbC3C_2Uo.mp4"
Deleting original file Key group behind Amazon Synod is funded by large U.S. pro-abortion charity-zVqbC3C_2Uo.f136.mp4 (pass -k to keep)
Deleting original file Key group behind Amazon Synod is funded by large U.S. pro-abortion charity-zVqbC3C_2Uo.f140.m4a (pass -k to keep)

(...Skipping 190+ videos that are already present in download log...)

[download] Downloading video 193 of 905
[youtube] xPL2y61Xj8M: Downloading webpage
[youtube] xPL2y61Xj8M: Downloading video info webpage
[download] Mother Miriam Live! 7.3.2019-xPL2y61Xj8M.mp4 has already been downloaded and merged
[download] Downloading video 194 of 905
[youtube] DzcQWkJ7sNA: Downloading webpage
[youtube] DzcQWkJ7sNA: Downloading video info webpage
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Mother Miriam Live! 7.2.2019-DzcQWkJ7sNA.mkv has already been downloaded and merged
[download] Downloading video 195 of 905
[youtube] zErHxgRDuow: Downloading webpage
[youtube] zErHxgRDuow: Downloading video info webpage
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Professor Roberto de Mattei explains the crisis in the Church - Part 2 of 2-zErHxgRDuow.mkv has already been downloaded and merged
[download] Downloading video 196 of 905

...and continues for 80 more videos, until Youtube blocks my IP address for a while.