Youtube download limited to 25 kb/s

[triplesixman]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• I'm reporting a bug unrelated to a specific site
• I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood

Since yesterday, youtube video downloads are all blocked at 25kb/s on my machines. The problem doesn't recur with a proxy. I didn't have any particular problems the days before. 25 kb/s without proxy, 4-5Mb/s with proxy

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• If using API, add 'verbose': True to YoutubeDL params instead
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

root@local:~/test# /usr/local/bin/yt-dlp.sh -vU  https://www.youtube.com/watch?v=6Dh-RL__uN4
[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=6Dh-RL__uN4']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (source)
[debug] Git HEAD: cc0619f62
[debug] Python 3.9.7 (CPython x86_64 64bit) - Linux-5.10.0-23-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1n  15 Mar 2022, glibc 2.31)
[debug] exe versions: ffmpeg N-111313-ge4d4d616ba-20230630 (setts), ffprobe N-111313-ge4d4d616ba-20230630, phantomjs 2.1.1
[debug] Optional libraries: sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
yt-dlp is up to date (stable@2023.07.06)
[youtube] Extracting URL: https://www.youtube.com/watch?v=6Dh-RL__uN4
[youtube] 6Dh-RL__uN4: Downloading webpage
[youtube] 6Dh-RL__uN4: Downloading ios player API JSON
[youtube] 6Dh-RL__uN4: Downloading android player API JSON
[youtube] 6Dh-RL__uN4: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] Testing format 616
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 27
[download] Destination: /tmp/tmprgueh0jf.tmp
[download] 100% of    712.00B in 00:00:01 at 676.50B/s
[info] 6Dh-RL__uN4: Downloading 1 format(s): 616+251
[debug] Invoking hlsnative downloader on "https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1690574704/ei/EMvDZM3rLubBsfIPkeukwAw/ip/ipv4here/id/e8387e44bfffb8de/itag/616/source/youtube/requiressl/yes/ratebypass/yes/pfa/1/wft/1/sgovp/clen%3D60390854%3Bdur%3D134.501%3Bgir%3Dyes%3Bitag%3D356%3Blmt%3D1663207460259625/hls_chunk_host/rr4---sn-q4flrnl7.googlevideo.com/mh/bk/mm/31,29/mn/sn-q4flrnl7,sn-q4fl6nss/ms/au,rdu/mv/m/mvi/4/pl/24/gcr/us/initcwndbps/73750/vprv/1/playlist_type/DVR/dover/13/txp/4532434/mt/1690552909/fvip/5/short_key/1/keepalive/yes/fexp/24007246,51000022/beids/24350017/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,pfa,wft,sgovp,gcr,vprv,playlist_type/sig/AOq0QJ8wRQIgMVWVx3BjGeGHVnF5bZheh7ePaKZWQe58T7oPtfJQVtECIQCCfKIOEITK-lIeqMszyyIcJH-nLNM2KUKZwvWtAZRJ9w%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AG3C_xAwRQIgHQxVwc0qTpbWozqPNzzHHaCJoiZmSMQAKAxbaHmxmJECIQCwLt0eZKrohEAKlAgGWAJpq_UQO2KWKzvyDHHXDdTcOA%3D%3D/playlist/index.m3u8"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 27
[download] Destination: bitch lasagna [6Dh-RL__uN4].f616.mp4
[download]  48.7% of ~ 327.14KiB at   26.57KiB/s ETA 00:06 (frag 2/27

[Hecker5556]

When I made my own youtube vid downloader (directly from the googlevideo links not manifest), and it was limited to 50kb/s per host, but then I did parallel downloading, and it seemed YouTube only limits speed for individual hosts not ip. if this issue becomes widespread then I suggest adding parallel downloading, which isn't heavy on memory if you iterate through small chunks. I recommend 10-20 parallel downloaders (using asyncio.gather(*tasks) for example)

edit: misread, saw proxy worked for you, I guess YouTube maybe blacklisted ur ip for too much scraping

[triplesixman]

I can provide access to allow you to reproduce the bug @pukkandan

As with the other nsig-related limitations (although that doesn't seem to be what you're talking about), it's a problem that only occurs for certain people and is difficult to reproduce. But in my case, the problem is systematic and I can provide the tools to enable you to check this.

[pukkandan]

It's a problem that only occurs for certain people and is difficult to reproduce.

This is all that the cant-reproduce label implies. Not that the issue is invalid

[RamsaTech]

@triplesixman Try this after getting a fresh ip from your ISP.

[absidue]

@triplesixman I'm guessing you are running yt-dlp on a server and mass downloading stuff?
If that's the case then the solution is to get a new IP, as YouTube is probably throttling your IP for doing too much.

[triplesixman]

@triplesixman Try this after getting a fresh ip from your ISP.

I have a fixed ip so it's not possible.

@triplesixman I'm guessing you are running yt-dlp on a server and mass downloading stuff? If that's the case then the solution is to get a new IP, as YouTube is probably throttling your IP for doing too much.

Same thing with my personal IP, unfortunately.

[RamsaTech]

Usually they ban the whole IP range, you should try using a mobile hotspot connection to test things out.

[triplesixman]

Usually they ban the whole IP range, you should try using a mobile hotspot connection to test things out.

Yup as shown under proxy it works. But I dare to hope that we can find a solution as for this similar topic I opened in the past ytdl-org/youtube-dl#29326

[RamsaTech]

Usually they ban the whole IP range, you should try using a mobile hotspot connection to test things out.

Yup as shown under proxy it works. But I dare to hope that we can find a solution as for this similar topic I opened in the past ytdl-org/youtube-dl#29326

Not sure what solution can be for an IP ban other then proxy/new ip !!

[pukkandan]

It is not a complete ban. They can still play on browser without throttling. So there is possibility of some solution. We are investigating the details in DMs. Will update when there's something concrete.

[triplesixman]

I was away for 3 days after this topic was created, and since my return I've had no more limitations from youtube. I will reopen a topic with more details if a new problem occurs. Thanks

[triplesixman]

Hello @pukkandan , it happens again, the limitation is 210kb/s (and no longer 25 kb/s as was the case a few days ago). On the browser (in private browsing), loading is also slow. If I'm connected to an google account, it's very fast.

PS C:\Users\Admin\Desktop\toast> C:\Users\Admin\Desktop\toast\yt-dlp_x86.exe https://www.youtube.com/watch?v=6Dh-RL__uN4 -vU
[debug] Command-line config: ['https://www.youtube.com/watch?v=6Dh-RL__uN4', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (win_x86_exe)
[debug] Python 3.7.9 (CPython AMD64 32bit) - Windows-10-10.0.22621-SP0 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
Current Build Hash: 7ce646283f2c437e2dc28e3008c892702824b6174f381d82b8e68c5bdb6be3b2
yt-dlp is up to date (stable@2023.07.06)
[youtube] Extracting URL: https://www.youtube.com/watch?v=6Dh-RL__uN4
[youtube] 6Dh-RL__uN4: Downloading webpage
[youtube] 6Dh-RL__uN4: Downloading ios player API JSON
[youtube] 6Dh-RL__uN4: Downloading android player API JSON
[youtube] 6Dh-RL__uN4: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] 6Dh-RL__uN4: Downloading 1 format(s): 22
[debug] Invoking http downloader on "https://rr4---sn-p5qlsn6l.googlevideo.com/videoplayback?expire=1693407423&ei=XwTvZO7BMpmjxN8PvbSk2AY&ip=myipv4&id=o-ALlQGsS_ip128vLrKDFEC7Gn0jE17dW7fZlnQzTYCqd5&itag=22&source=youtube&requiressl=yes&mh=bk&mm=31%2C29&mn=sn-p5qlsn6l%2Csn-p5qddn7r&ms=au%2Crdu&mv=m&mvi=4&pl=25&gcr=us&initcwndbps=1402500&spc=UWF9f6IL2pERGZGHFRMjaiA6shjMMq4&vprv=1&svpuc=1&mime=video%2Fmp4&cnr=14&ratebypass=yes&dur=134.559&lmt=1655016664588351&mt=1693385358&fvip=1&fexp=24007246%2C51000024&c=ANDROID&txp=4532434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAK0FM3J0K-CQ_MqGxxjWh6BQ5vuB6-0v-uUAo_-ceHFXAiBdDzglG9CRFOwHW4nkmcLLEIkysJ3WmmeI5ffMKccyVw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAPw6LEb6CHtL68QrDoFCn3z_T7IFzh-141kBJGUspk3uAiEA8IUgbxEKpURDeRCZ_z2ADECxUVOSr3oST4TSARgc1RU%3D"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: bitch lasagna [6Dh-RL__uN4].mp4
[download] 100% of   14.10MiB in 00:01:08 at 209.62KiB/s

If needed, the contents of the command with --dump-pages.

dump_pages.txt

[xecus922]

Just wanted to add that this has also been happening to me for at least a few months now sadly. Definitely being throttled as download speed from other sites is much, much higher, much more reasonable speeds. Limited to just about the same speed as triplesixman.

Thanks for making this post and others here trying to help find a solution.

[xecus922]

Was trying to edit previous post with some hopefully useful additional information but the edit isn't saving for some reason.

But just some things I wanted to add.

I think it may be possible to reproduce for everyone, you just need to scrape enough for it to happen to you lol

We just don't know what exactly that 'threshold' (or thresholds) is to trigger that 'flag' in their system to throttle the user.

And then there may be exceptions for certain locations where they're more lax and lenient before the throttles kick in.

Remember this is YouTube, with some very impressive algorithms for things like recommended videos, it's scary how good it is.

So this throttling system of theirs could be pretty complex. Notice how triplesix's speeds actually started to increase after not downloading for a few months then the throttle returned shortly after, which was very interesting and it's hard to make sense of why it works like that.

So it could be some kind of 'monitoring' going on for previously throttled IPs for whatever reason, maybe they're teaching the system, who knows.

It's not crazy to speculate that they may have some kind of system set up to watch certain users who may be scraping for what could be months before taking action and throttling, I've seen it being done in online games with very small communities of just a few thousand with the lead developer and creator of the game admitting it himself.

But I can tell you that the throttle seems to kick in after scraping thousands of videos or maybe just X amount of bandwidth usage in Y time.

It's definitely a lot compared to the average yt-dl user probably so that may be why some are finding it difficult to reproduce in a short amount of time.

It took me months of daily archiving for the throttles to hit me whereas for others the throttle may kick in much sooner for whatever reason.

[triplesixman]

Use aria2 to download parallel splits:

--downloader aria2c --downloader-args 'aria2c:--continue --max-concurrent-downloads=30 --max-connection-per-server=16 --split=30 --min-split-size=1M'

Still limited to 30kb/s. Currently yt-dlp is no longer functional on my side with YouTube, even when changing IP (proxy or VPN), I sometimes get a 403 error.

[onyxcode]

Use aria2 to download parallel splits:

--downloader aria2c --downloader-args 'aria2c:--continue --max-concurrent-downloads=30 --max-connection-per-server=16 --split=30 --min-split-size=1M'

this worked for me thank you!