Add --html=$html option to solve the "429 too many request" issue

[TheGr33k]

I think i have though of a durable solution to solve the "429 too many request" issue.Suppose i can get the html for the page by any other means (like using my ip addresses binded to my network interface) , i can add the html as a parameter in the youtube-dl is present.If the html is present as an option , then youtube-dl won't have to fetch html once again resulting in fewer request to youtube and thus less chance of getting the error.On top of that its increased performance for youtube-dl since it has less work to do to generate the direct youtube url.It should be too hard i think

Moreover you can add the throttle feature as well to further foolproof youtube-dl it may help in the 429 issue

Check #4550

[TheGr33k]

Anyone could look at it please .....

[phihag]

This approach could not possibly work request information from multiple URLs (HTML or other). Therefore, I see two possible solutions:

• Implement IP binding right in youtube-dl
• Allow using external downloaders

[TheGr33k]

I don't understand why its not possible.The approach is rather simple, let me explain once again.Currently youtube-dl fetch youtube html page and then extract the direct url/download depending on options

My idea is that if you add an option so that we can feed the html to youtube-dl , youtube-dl just have to extract the direct url/download depending on options

IP binding is also awesome but since many people have asked for it and its hasn't been done yet(i assume its a tough job) the html option would allow people to mimic the ip binding by fetching the html of the page via curl using an ip address of their choice and then feeding it to youtube-dl

[jaimeMF]

And you are assuming that only one webpage is needed for extracting the info, but that's not true. In the case of YouTube: we need the actual video webpage and the dash manifest (which contains some additional formats), and for age restricted videos we need an additional page.

[TheGr33k]

Yes i was assuming that only one page is needed ........Philip can you
please add the bind ip feature asap(at least for the g option).The 429 too
many request is really nagging me here

On Sat, Jan 10, 2015 at 2:07 PM, Jaime Marquínez Ferrándiz <
notifications@github.com> wrote:

And you are assuming that only one webpage is needed for extracting the
info, but that's not true. In the case of YouTube: we need the actual video
webpage and the dash manifest (which contains some additional formats), and
for age restricted videos we need an additional page.

—
Reply to this email directly or view it on GitHub
#4551 (comment).

[phihag]

Starting from youtube-dl 2015.01.10.1, you can make use of the --source-address option. See our FAQ if you need help updating.

We may implement external downloaders too, but I do think the original issue has been solved.

[TheGr33k]

I think there is a bug in the source ip address feature

youtube-dl -g --source-address 209.96.45.3 https://www.youtube.com/watch?v=duKL2dAJN6I

[root@485067 ~]# youtube-dl -g --source-address 209.95.42.7 https://www.youtube.com/watch?v=duKL2dAJN6I
https://r4---sn-a5m7zne6.googlevideo.com/videoplayback?ipbits=0&mm=31&signature=1E1A63155E54F8CE3230A8018D811CFFB6D3661C.0451523EA0587250CECCB170C9DB7B7DC1E1B3E6&source=youtube&mv=m&dur=772.295&initcwndbps=2420000&itag=22&ms=au&upn=J5PcxI92fOY&fexp=900718%2C927622%2C932404%2C9405627%2C941004%2C943917%2C947209%2C947218%2C947225%2C948124%2C949429%2C952302%2C952605%2C952901%2C955301%2C957103%2C957105%2C957201%2C959701&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmime%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&id=o-AIe3FMhwkaSAn05NxCULKPKDgxEMnHCyMHd9FaVJN49N&mt=1420917791&requiressl=yes&mime=video%2Fmp4&ratebypass=yes&expire=1420939413&sver=3&ip=206.190.143.22&key=yt5

Notice that the source address and &ip= in output isn't same.

[phihag]

As mentioned in our contribution guidelines, please do not post about multiple different issues in a single issue entry.

Instead, create new issue entries for new issues. Do not worry about creating issues; it is free of charge (and allows us to keep track which problems have been solved and which have not been).

As also mentioned in the bug reporting guidelines, unless you are certain that it is not necessary, please include the output you get when you add -vC to your command line (using youtube-dl 2015.01.10.2 or newer which include better debugging capabilities). In your case, a pcap dump would also be extremely helpful.