Why is the speed of downloading not consistent?

[gbraad]

Checklist

• I'm asking a question
• I've looked through the README and FAQ for similar questions
• I've searched the bugtracker for similar questions including closed ones

Question

When using a browser (or any HTTP) download or watching a video on youtube, I notice the download is very consistent... mostly peaking at the bandwidth. However, when I use youtube-dl to grab a video, it shows a very clear roof-tiling pattern; it goes up and when it reaches peak there is a sheer drop to almost zero... after which the speed starts to raise again and repeating this process.

> youtube-dl --version
2019.09.28

Not seeing this using using Aria2 (on Windows) as an external downloader:

PS> choco install aria2
PS> youtube-dl --external-downloader aria2c --external-downloader-args "-j 12 -s 12 -x 12 -k

Ref #22652
Note: I am aware youtube-dl does not perform any traffic shaping or throttling, but since this is not noticeable with an external downloader, it might be an issue with this specific version on Windows. This was not noticeable in previous versions.

[dstftw]

Post complete verbose log of command you are running.

[gbraad]

This is a question and NOT a bug report yet, so this is not in your Issue template. I am more than willing to provide this, but tagging 'incomplete' or 'locking conversations' is not helpful.

[gbraad]

C:\Users\gbraad\Videos>youtube-dl -v https://www.youtube.com/watch?v=7IvmzEuVkSY
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=7IvmzEuVkSY']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.6.0 (CPython) - Windows-10-10.0.18995-SP0
[debug] exe versions: ffmpeg N-91378-g3f953379e1, ffprobe N-91378-g3f953379e1
[debug] Proxy map: {}
[youtube] 7IvmzEuVkSY: Downloading webpage
[youtube] 7IvmzEuVkSY: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://r3---sn-a5mekn7d.googlevideo.com/videoplayback?expire=1570629326&ei=bpKdXcq8Kovikgav1aDoAg&ip=155.94.198.5&id=o-ANBVGwM8kk9oMBX5ZkYIYY-O4kcS78eUzoYla5TyjZHT&itag=298&aitags=133%2C134%2C135%2C136%2C160%2C242%2C243%2C244%2C247%2C278%2C298%2C302&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-a5mekn7d%2Csn-a5mlrn7s&ms=au%2Crdu&mv=m&mvi=2&pl=23&initcwndbps=903750&mime=video%2Fmp4&gir=yes&clen=1300773514&dur=4730.266&lmt=1570497695701219&mt=1570607624&fvip=6&keepalive=yes&fexp=23842630&c=WEB&txp=5432432&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRQIhAOrIJAmC74xnGH03g5P0ASyaQeZkGKsPaDFmrG7zWUbdAiBFEnLS8wOA_kVzhYIm5dWvpmrDkUlI_dFHdoyMbuP8JQ%3D%3D&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRQIhAKzlWTXWLVxckJwnogMOoRtRRckhSth09a0BSe5ti28gAiBRDpLU8INYLIwmxHLcAM-1Ba7yTbrbcK6T1u7je3kNkA%3D%3D&ratebypass=yes'
[download] Resuming download at byte 1300773514
[download] Buster Keaton - The General 1926 Colorized with Deep Learning AI-7IvmzEuVkSY.f298.mp4 has already been downloaded
[download] 100% of 1.21GiB
[debug] Invoking downloader on 'https://r3---sn-a5mekn7d.googlevideo.com/videoplayback?expire=1570629326&ei=bpKdXcq8Kovikgav1aDoAg&ip=155.94.198.5&id=o-ANBVGwM8kk9oMBX5ZkYIYY-O4kcS78eUzoYla5TyjZHT&itag=251&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-a5mekn7d%2Csn-a5mlrn7s&ms=au%2Crdu&mv=m&mvi=2&pl=23&initcwndbps=903750&mime=audio%2Fwebm&gir=yes&clen=83939556&dur=4730.381&lmt=1570488278406262&mt=1570607624&fvip=6&keepalive=yes&fexp=23842630&c=WEB&txp=5411222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRgIhANZfoZJlBH4eNDWEyxNaaf67vPRxfTI2Ie_lMNFy0jeNAiEAtlWGQppiVZRQA7TpaIo_f5U4Uwrl_yYQ4Mie0dzUSmc%3D&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRQIhAKzlWTXWLVxckJwnogMOoRtRRckhSth09a0BSe5ti28gAiBRDpLU8INYLIwmxHLcAM-1Ba7yTbrbcK6T1u7je3kNkA%3D%3D&ratebypass=yes'
[download] Destination: Buster Keaton - The General 1926 Colorized with Deep Learning AI-7IvmzEuVkSY.f251.webm
[download]  27.3% of 80.05MiB at  1.37MiB/s ETA 00:42
ERROR: Interrupted by user

[dstftw]

Youtube shapes bandwidth for audio-only and video-only formats after first ~10MiB. In order to bypass that connection is reopened each 10MiB.

[gbraad]

Thanks for the explaination, @dstftw Is there a way around this, like fetching a combined file instead? (prefer to use a single solution, instead of having to rely on an external downloader).