YouTube: remove "sig" detection

[ghost]

"sig" detection was added back in 2012:

http://github.com/rg3/youtube-dl/commit/9ca6670

in response to this issue:

http://github.com/rg3/youtube-dl/issues/427

However the video in that issue, and all videos from what I can tell, no longer
utilize "sig" type protection. From my testing everything seems to use the
normal "signature" or "s" type protection. Here is an example of the "problem"
code in question:

http://github.com/rg3/youtube-dl/blob/9da526a/youtube_dl/extractor/youtube.py#L1520-L1521

Now it doesnt really hurt to leave it in, but I feel that if it is to stay we
should have at least 1 example video that uses it.

[yan12125]

That's a development challenge of youtube-dl - we never know whether an API is still usable or not. If it does not bring problems, just keep it. I'm afraid removing this breaks lots of usages. If you can get a response from YouTube engineers stating that 'sig' is removed from their codebase, I'll remove it from youtube-dl as well.

[ghost]

@yan12125 I strongly disagree with your opinion. This checking is in place, yet you have not a single example video in the test suite that utilizes it? Does not make sense

[yan12125]

YouTube may use different mechanisms for signatures, depending on the video, the user agent string, or random factors. My idea is if nothing is broken, just keep it. Of course it's difficult to have test cases covering all possible cases.

[ghost]

Good point. While where at it, lets add a bunch of if (0), cause you never know when 0 might equal 1!

[yan12125]

Here's a (beautified) fragment from https://s.ytimg.com/yts/jsbin/player-en_US-vflL5aRF-/base.js, the main script of HTML5 player from https://www.youtube.com/watch?v=yZIXLfi8CZQ:

        Qx = function(a, b) {
            for (var c = [], d = 0; d < b.length; d++) {
                var e = b[d];
                if (e.sig || e.s) {
                    var f = e.sig || $r(e.s);
                    e.url = xj(e.url, {
                        signature: f
                    })
                }
                e.url && c.push(Ox(e.url, e.type, e.quality, e.itag, e.width, e.height, !!e.isFastLoad))
            }
            return fx(a.G, c)
        },

As you can see the YouTube HTML5 player still handles sig field, so we should do the same thing in youtube-dl.

[yan12125]

By the way https://www.youtube.com/watch?v=yZIXLfi8CZQ breaks youtube-dl. I'll try to solve it.

[ghost]

Thank you for the investigation, that is interesting what you found. However just because youtube-dl and even YouTube check for "sig", doesnt mean that it is something that should be checked. Unless at least a single example exists that utilizes "sig", there is no point to check it.

While we are at it, we might as well check for e.monkey or e.banana

[yan12125]

Sounds strange to me. If YouTube checks sig, it should be checked in youtube-dl, too. A design goal of youtube-dl is mimicking what browsers/mobile apps do. (Of course there's no guarantee to 100% compatibility due to limited time and technical restrictions.)

You can ask YouTube engineers to remove the redundant check or provide an example with sig. It's not youtube-dl's fair.

[ghost]

@yan12125 you are mistaken. It is you that needs to provide an example of sig. My position is that no such example exists, so why/how would I provide one?

[yan12125]

As I've already said there's no need to provide an example anymore. YouTube checks sig. For youtube-dl that's enough.

BTW, I'm not asking you to provide an example. I'm asking you to ask YouTube for an example. Sorry if my English is misleading.