Proxy only when needed (Global feature)

[spookyahell]

Please follow the guide below

• You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
• Put an x into all the boxes [ ] relevant to your issue (like that [x])
• Use Preview tab to see how your issue will actually look like

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.08.28. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2016.08.28

Before submitting an issue make sure you have:

• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

• Single video: http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/

---

Description of your issue, suggested solution and other information

It would be nice to be able to set a proxy that only gets active when a regular request fails.
So, shall any request be getting an error 403 then it would repeat the request with proxy instead of outputting error imediatly. Option could be something like --proxy-on-occasion and -proxoc and then the HTTP-Proxy Link. This would need to be global for the entire application. By default 403 should result in a re-request of the "Forbidden" object. And then there could be a comma seperated list option like this --proxoc-error-types and a valid value would be 403,404,503

[yan12125]

Not quite understand the motivation. Why not just add --proxy if you know a video is geo-restricted?

[spookyahell]

Because the proxy itself currently doesn't support all URLs but frankly it needs a proxy. But theres a proxy already setup in the network, but it also doesn't support them all, so it's like fail standard then use this proxy, it's complicated.

[yan12125]

Sounds like --geo-verification-proxy or a Proxy Auto Config (PAC) script is what you need. The latter is requested in #8278, and I'm working on it recently. There are several requests involved in http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/. Can you elaborate what kind of URLs should go to the first proxy and what should go to the second?

[spookyahell]

Well, urls with link.theplatform need to go reguarly and ipad-streaming.cbs.com need to go via the proxy specified wherever.

[yan12125]

OK PAC scripts should fit your need. Please move to #8278 for further discussion.

[dstftw]

PAC script won't help when same host serves geo restricted videos along with non geo restricted.

[yan12125]

FindProxyForURL, the main function in PAC scripts, accepts two arguments url and host. As long as the restriction is determined by URL, PAC scripts can handle it.

[dstftw]

Most of geo restrictions I've seen have no special pattern for geo restricted videos thus managing rules per URL is not feasible. It's much easier to simply pass --proxy in this case rather than bother writing a rule for exact URL that will only be used once.

[yan12125]

Did you mean some videos on link.theplatform.com are geo-restricted while others not? My idea is that all requests to link.theplatform.com go through a proxy, no matter it's geo-restricted or not. A browser plugin Unblock-Youku uses PAC scripts to bypass geo-restrictions on several Chinese websites. If it still sounds bad, please go ahead and reopen this issue.

Note that for http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/, the key request http://link.theplatform.com/s/dJ5BDC/media/guid/2198311517/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3?mbr=true&manifest=m3u&format=SMIL always returns HTTP 200 OK, so the proposed approach does not work. A possible solution may be adding a new Exception class for geo-restriction errors, just like #10308 for authentication.

[dstftw]

I wasn't concretely about theplatform. What I was trying to say is that if say there is an API URL http://foo.bar/video/<id> where we put video id to fetch video data and this URL succeeds for non-restricted videos and fails with 403 for restricted videos then there is no way to figure out beforehand whether access to this URL needs to be proxied or not as there is not special pattern in URL that allows to distinguish them.

[yan12125]

Maybe I wasn't clear. My proposal is: proxy all such API requests, no matter the corresponding video is geo-restricted or not. That is, assume an API request always requires proxy. What PAC scripts do is distinguishing URLs that may require proxy.