Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
[YouTube] Signature videos return 403 forbidden sporadically from multiple locations #24713
Labels
Comments
This was referenced Apr 10, 2020
|
|
Same problem and yet your topic has also been closed lol |
|
Results seem completely random on signature videos. Sometimes it completes and merges normally and sometimes it gets 403 on the invoke download. It is almost certainly a signature issue (and new) and if Sergey refuses to acknowledge it I will try to fix it myself.
…On Apr 10, 2020, 6:14 AM -0400, titirino159 ***@***.***>, wrote:
Same problem and yet your topic has also been closed lol
Is it random? If you retry the same download a few seconds later, does it work?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Sergey - are you trying to say this is somehow python 3.8 related? Is that why your responses specifically use 3.7? Did you try each video 10+ times to see if you can reproduce the random errors?
I’ve been using YouTube-dl with 3.8 for 4 months without issue.
…On Apr 10, 2020, 6:14 AM -0400, titirino159 ***@***.***>, wrote:
Same problem and yet your topic has also been closed lol
Is it random? If you retry the same download a few seconds later, does it work?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Seems like they are also working on this issue here: Tyrrrz/YoutubeExplode#349 (cc: @dstftw)
I’m finding some tickets on every YouTube downloading project.
At first they thought it was just an age-gate issue before realizing it’s happening on non age-gated videos.
…On Apr 10, 2020, 8:08 AM -0400, Mazin Khoury ***@***.***>, wrote:
Sergey - are you trying to say this is somehow python 3.8 related? Is that why your responses specifically use 3.7? Did you try each video 10+ times to see if you can reproduce the random errors?
I’ve been using YouTube-dl with 3.8 for 4 months without issue.
On Apr 10, 2020, 6:14 AM -0400, titirino159 ***@***.***>, wrote:
> Same problem and yet your topic has also been closed lol
> Is it random? If you retry the same download a few seconds later, does it work?
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Can you rule out something interfering on your local side? |
|
I've been able to reproduce it on my MBP locally as well as in 2 different data centers on Ubuntu. |
|
Yeah, data centers might cause issues, not because of a blanket IP address ban, but they can be identified (depending on their size, and peering etc.) by their ASN, and Google/YT definitely handles incoming connections differently depending on ASN. |
|
Right, I understand. But why would I be able to reproduce it locally on my home ISP? Also, it's only happening on signature videos and is not consistent (sometimes it works, sometimes it doesn't). |
|
Yeah residential connections ("normal" ISPs) should definitely work.. Not sure what you mean with signature videos, but if you have a link to a video right now that is not working for you, I can test that quickly here on my end. |
|
Here is a better illustration of the randomness: All 3 are from the same location and virtualenv. Just running the command 3 times. By videos with a signature, I mean videos that have encrypted signatures that are decrypted by youtube-dl and then added to the "invoking download" link. There is clearly something inconsistent with the signatures (recently changed on youtubes side) which is causing inconsistencies in the creation of the download link. @dstftw explanation for this? |
|
I think I'm on to something... In these above downloads, we are trying to grab f248 + f251 and then merge. When we are assessing the signature length, if the length of f248 and f251 are both 112, the download completes normally. In the instance where it fails, it fails on f251 which has a signature length of '108' instead of '112'. Anytime it attempts to download a format with a 108 signature length, the download fails. Whether the signature length of any given format is '112' or '108' seems to be random and changes each time I attempt to download the file. It is only successful when ALL the formats you are requesting have a signature length of 112 and fail anytime any has a length of 108. Please re-open this issue @dstftw. I have now reproduced it on several videos. |
|
Reproducing on 2nd video with different formats: First run successful, both requested formats (f137 and f251) have signature length 112. On the 2nd attempt, it fails because the first requested format (f137) has a signature length of 108. |
|
Okay, quick attempt for those IDs: PS C:\Move> youtube-dl --ignore-config 2t22OYi1F7E
[youtube] 2t22OYi1F7E: Downloading webpage
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Destination: These Days-2t22OYi1F7E.f137.mp4
[download] 100% of 5.00MiB in 00:00
[download] Destination: These Days-2t22OYi1F7E.f251.webm
[download] 100% of 3.19MiB in 00:00
[ffmpeg] Merging formats into "These Days-2t22OYi1F7E.mkv"
Deleting original file These Days-2t22OYi1F7E.f137.mp4 (pass -k to keep)
Deleting original file These Days-2t22OYi1F7E.f251.webm (pass -k to keep)
PS C:\Move> youtube-dl --ignore-config EwM8Y7h2TcA
[youtube] EwM8Y7h2TcA: Downloading webpage
[download] Destination: Reset - Tiger J.K (ft. Jinshil of mad soul child) Han_Rom_Eng Lyrics [Who Are You School 2015 OST]-EwM8Y7h2TcA.f248.webm
[download] 100% of 9.23MiB in 00:01
[download] Destination: Reset - Tiger J.K (ft. Jinshil of mad soul child) Han_Rom_Eng Lyrics [Who Are You School 2015 OST]-EwM8Y7h2TcA.f251.webm
[download] 100% of 3.83MiB in 00:00
[ffmpeg] Merging formats into "Reset - Tiger J.K (ft. Jinshil of mad soul child) Han_Rom_Eng Lyrics [Who Are You School 2015 OST]-EwM8Y7h2TcA.webm"
Deleting original file Reset - Tiger J.K (ft. Jinshil of mad soul child) Han_Rom_Eng Lyrics [Who Are You School 2015 OST]-EwM8Y7h2TcA.f248.webm (pass -k to keep)
Deleting original file Reset - Tiger J.K (ft. Jinshil of mad soul child) Han_Rom_Eng Lyrics [Who Are You School 2015 OST]-EwM8Y7h2TcA.f251.webm (pass -k to keep)
PS C:\Move>Seems to work.. youtube-dl stores extraction signature stuff into a cache directory (usually inside |
|
Interesting - I had a different cache_spec for 108 for that player. After deleting the /.cache/youtube-dl directory, and re-downloading 2 players, it seems to have fixed the issue. |
|
#23638 |
|
This issue is still being reported constantly (there are dozens of bug reports on it closed and open). It's being reported on Reddit (e.g. here) and I'm seeing it myself. Clearing the cache does indeed fix it (but for how long?), but there is no reason for an end-user to know that, and it also certainly shows that youtube-dl is handling something in that cache directory incorrectly, or should be able to detect whatever data is in there that is "bad". This should be reopened because the underlying bug is not fixed, it's just individual users who stumble in here are fixing it themselves. |
|
Just wanted to follow up - this issue recurred again today roughly 1 week after clearing the cached sigfuncs folder. Same fix resolved the issue. Since dstftw refuses to consider this an issue, a temporary solution is to set a cron deleting |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Checklist
Verbose log
Another location and video:
Description
I understand you are used to seeing many 403 bug reports that are just throttling/client side/provider site issues. This is NOT a duplicate and only seems to be happening in certain videos with signatures. The videos are all available publicly without an account from my location.
I am experiencing this from multiple locations and environments. It is a new issue that I only started noticing in the last week and it is not consistent. If you attempt the same file over and over you will see mixed successes and errors.