KeyError: 'sig' on any youtube video #2497

dingens · 2014-03-03T00:41:50Z

I cannot download youtube videos. On really everyone (I even tried some from the front page to verify) I get this:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/bin/youtube-dl/__main__.py", line 7, in <module>
  File "/usr/bin/youtube-dl/__init__.py", line 536, in main

  File "/usr/bin/youtube-dl/__init__.py", line 520, in _real_main

  File "/usr/bin/youtube-dl/FileDownloader.py", line 475, in download
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 80, in extract
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 405, in _real_extract
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 405, in <genexpr>
KeyError: 'sig'

Version: both 2012.09.27 (debian wheezy) and the newest one (2014.02.28)

jaimeMF · 2014-03-03T07:14:23Z

Could you post the command line you are using and the full output you get when running youtube-dl --verbose YOUR_URL?
The output you have posted is from an old version.

astazed · 2014-03-03T10:53:49Z

I noticed that they replaced the 'sig' parameter with 'signature'

so this line in youtube.py:

if 'sig' in url_data:
    url += '&signature=' + url_data['sig'][0]

should be replaced by:

if 'signature' in url_data:
    url += '&signature=' + url_data['signature'][0]

I am still receiving sometimes 'sig' instead of 'signature' so the best would be to check for both.
That is what I am doing in java:

String sig = paramsMap.get("signature");
if(sig == null){
    sig = paramsMap.get("sig");
    if(sig == null){
        sig = decrypt_signature(paramsMap.get("s"), videoUrl);
    }
}

jaimeMF · 2014-03-03T12:16:06Z

Could you please post the full output you get when running youtube-dl --verbose YOUR_URL?, like:

$ youtube-dl 'http://www.youtube.com/watch?v=UxxajLWwzqY' --verbose
[debug] System config: []
[debug] User config: ['--keep-video', '--write-info-json', '--write-thumbnail', '--verbose']
[debug] Command-line args: ['http://www.youtube.com/watch?v=UxxajLWwzqY', '--verbose']
[debug] Encodings: locale 'UTF-8', fs 'utf-8', out 'UTF-8', pref: 'UTF-8'
[debug] youtube-dl version 2014.02.28
[debug] Git HEAD: 5a0b262
[debug] Python version 3.3.4 - Darwin-13.1.0-x86_64-i386-64bit
[debug] Proxy map: {}
[youtube] Setting language
[youtube] UxxajLWwzqY: Downloading webpage
[youtube] UxxajLWwzqY: Downloading video info webpage
[youtube] UxxajLWwzqY: Extracting video information
[youtube] UxxajLWwzqY: Encrypted signatures detected.
[youtube] encrypted signature length 87 (42.44), itag 22, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 43, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 18, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 5, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 36, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 17, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 137, html5 player ima-vfl4_saJa
[youtube] UxxajLWwzqY: Downloading js player vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 136, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 135, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 134, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 133, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 160, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 140, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 84 (41.42), itag 171, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 137, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 136, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 135, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 134, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 133, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 160, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 140, html5 player ima-vfl4_saJa
[youtube] encrypted signature length 87 (42.44), itag 171, html5 player ima-vfl4_saJa
[info] Writing video description metadata as JSON to: Icona Pop - I Love It (feat. Charli XCX) [OFFICIAL VIDEO]-UxxajLWwzqY.info.json
[youtube] UxxajLWwzqY: Downloading thumbnail ...
[youtube] UxxajLWwzqY: Writing thumbnail to: Icona Pop - I Love It (feat. Charli XCX) [OFFICIAL VIDEO]-UxxajLWwzqY.jpg
[download] Resuming download at byte 1047552
[download] Destination: Icona Pop - I Love It (feat. Charli XCX) [OFFICIAL VIDEO]-UxxajLWwzqY.mp4
[download]   4.8% of 57.95MiB at 803.11KiB/s ETA 01:10

As you can see this video works.

dingens · 2014-03-03T13:03:49Z

Sorry, I forgot the ./ when trying the new version ;) With the new version it works, sorry... Here's the verbose output but as far as I can see there's no difference:

/tmp$ /usr/bin/youtube-dl http://www.youtube.com/watch?v=UxxajLWwzqY --verbose
[debug] Proxy map: {}
[youtube] Setting language
[youtube] UxxajLWwzqY: Downloading video webpage
[youtube] UxxajLWwzqY: Downloading video info webpage
[youtube] UxxajLWwzqY: Extracting video information
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/bin/youtube-dl/__main__.py", line 7, in <module>
  File "/usr/bin/youtube-dl/__init__.py", line 536, in main

  File "/usr/bin/youtube-dl/__init__.py", line 520, in _real_main

  File "/usr/bin/youtube-dl/FileDownloader.py", line 475, in download
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 80, in extract
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 405, in _real_extract
  File "/usr/bin/youtube-dl/InfoExtractors.py", line 405, in <genexpr>
KeyError: 'sig'

btw: quote

if 'signature' in url_data:
    url += '&signature=' + url_data['signature'][0]

shouldn't that be escaped (for the case that signature contains percent signs, &s and the like)?

jaimeMF · 2014-03-03T13:08:13Z

You are clearly running an outdated version: it should print more [debug] * info and the InfoExtractors.py and FileDownloader.py files are not used anymore.
We haven't found any signature with those characters, they only contain alphanumeric characters and dots.
If you can reproduce the error with the last version, post the output and we will reopen the issue.

astazed · 2014-03-03T13:46:57Z

Am I the only one getting url_encoded_fmt_stream_map like the following?

stream_map": "fallback_host=tc.v22.cache3.googlevideo.com&url=http://r1---sn-gxo
5uxg-jqbe.googlevideo.com/videoplayback?mv=m&ratebypass=yes&ip=130.120.247.174&m
s=au&itag=43&sparams=id%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpir
e&expire=1393877846&upn=i7irEM5SKVY&key=yt5&id=4f14b1a072ac4798&mt=1393853634&sv
er=3&source=youtube&fexp=902904%2C916917%2C910207%2C942701%2C937417%2C937416%2C9
13434%2C936910%2C936913%2C902907&ipbits=0&signature=C01BACD67C4C18D54ECE82A9A5DC
F86459CE6C04.046E53CD9BABEF3F44442AF6765829792D232909&quality=medium&itag=43&typ
e=video/webm; codecs="vp8.0, vorbis"
fallback_host=tc.v22.cache3.googlevideo.com&url=http://r1---sn-gxo5uxg-jqbe.goog
levideo.com/videoplayback?mv=m&ratebypass=yes&ip=130.120.247.174&ms=au&itag=18&s
params=id%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&expire=13938
77846&upn=i7irEM5SKVY&key=yt5&id=4f14b1a072ac4798&mt=1393853634&sver=3&source=yo
utube&fexp=902904%2C916917%2C910207%2C942701%2C937417%2C937416%2C913434%2C936910
%2C936913%2C902907&ipbits=0&signature=BE2D7BB23F1300CA74DB9CF45BDAC72035CB9EB3.0
F374AE667EB5445F236EB7F4B162A8678A9B278&quality=medium&itag=18&type=video/mp4; c
odecs="avc1.42001E, mp4a.40.2"     ...

where
signature=BE2D7BB23F1300CA74DB9CF45BDAC72035CB9EB3.0F374AE667EB5445F236EB7F4B162A8678A9B278

are replacing sig= ....

jaimeMF · 2014-03-03T13:51:38Z

@astazed I get s=* in https://www.youtube.com/watch?v=UxxajLWwzqY, if youtube-dl fails please open a new issue with the verbose output.

dingens · 2014-03-03T14:09:53Z

Yes, as I said, I confused them yesterday. With the 2014 one it works, so there'd be no use of posting the verbose output, so I pasted the one from the 2012 version.

I'll use the new version now, but as the 2012.something version is the one included in Debian stable, it'd be nice to update that one, too. I don't know how that is done, maybe it's better to file a bug over at debian?

phihag · 2014-03-03T14:44:23Z

Sure, feel free to file a bug report at debian. However, we do not maintain older versions. Since video services change regularly, there's really no point in separating features and bugfixes, since an inordinate amount of changes are what would be considered bug fixes at other projects.

dingens · 2014-03-03T16:40:48Z

Yes that's right. I'll leave that up to the maintainer. I remember that there have been updates of that package in the past in case of such problems rendering the tool useless. We'll see. Thanks so far!

rbrito · 2014-03-03T17:22:26Z

@Maix, you should really take the packages that I upload to unstable. Despite the name, unstable is not something that you should fear and, in fact, that's essentially the only distribution that I care about (any potential backport was simply not done or requested by me).

Th testing distribution is also "supported" (in the sense that I will care about it), but I think that I will never let a youtube-dl version be released in a stable release (I would have too many problems getting it uploaded/reviewed by the Stable Release Managers, as they would complain that youtube-dl's code changes way too much---and, in fact, it does, due to the very nature of scraping tools and web services).

dingens · 2014-03-03T18:10:30Z

I just saw that I don't have the version installed from wheezy but still one from wheezy and that it isn't included in wheezy at all. That means we can close the debian bug I think?

Your suggestion would mean one had to include unstable in the sources.list all the time, which isn't all that easy for normal users... But as long as stable remains that strict without exceptions for highly "api-bound" applications there's probably no way to solve that!?

The only way to solve that would be to really separate the plugins from the core and sandbox them (or define some language for the parsing process instead of using python), so that those can be updated independently, maybe even by some third-party site as it is done by greasemonkey or stylish. That'd really be cool but I think it's too complicated(?)

jaimeMF closed this Mar 3, 2014

ytdl-org / youtube-dl

KeyError: 'sig' on any youtube video #2497

KeyError: 'sig' on any youtube video #2497

dingens commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

astazed commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

dingens commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

astazed commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

dingens commented Mar 3, 2014

phihag commented Mar 3, 2014

dingens commented Mar 3, 2014

rbrito commented Mar 3, 2014

dingens commented Mar 3, 2014

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

KeyError: 'sig' on any youtube video #2497

KeyError: 'sig' on any youtube video #2497

Comments

dingens commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

astazed commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

dingens commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

astazed commented Mar 3, 2014

jaimeMF commented Mar 3, 2014

dingens commented Mar 3, 2014

phihag commented Mar 3, 2014

dingens commented Mar 3, 2014

rbrito commented Mar 3, 2014

dingens commented Mar 3, 2014

Essential cookies

Always active

Analytics cookies