Can't download from twitch.tv

[suckmyglock]

Using latest mpv and youtube-dl.
Running windows, unsure how to post log

[lemmi]

experiencing the same thing, livestreamer and a kodi plugin don't work either. so i guess they changed api.

$ mpv http://twitch.tv/twitchplayspokemon --ytdl-raw-options=verbose=
Playing: http://twitch.tv/twitchplayspokemon
[ytdl_hook] [debug] System config: []
[ytdl_hook] [debug] User config: []
[ytdl_hook] [debug] Command-line args: [u'--no-warnings', u'-J', u'--flat-playlist', u'--sub-format', u'ass/srt/best', u'--no-playlist', u'--format', u'bestvideo[height <=? 1440]+bestaudio/best', u'--verbose', u'--all-subs', u'--', u'http://twitch.tv/twitchplayspokemon']
[ytdl_hook] [debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8
[ytdl_hook] [debug] youtube-dl version 2016.09.11
[ytdl_hook] [debug] Python version 2.7.12 - Linux-4.7.3_1-x86_64-with-glibc2.2.5
[ytdl_hook] [debug] exe versions: ffmpeg 2.8.7, ffprobe 2.8.7, rtmpdump 2.4
[ytdl_hook] [debug] Proxy map: {}
[ytdl_hook] ERROR: Unable to download JSON metadata: HTTP Error 400: Bad Request (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[ytdl_hook]   File "/usr/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 394, in _request_webpage
[ytdl_hook]     return self._downloader.urlopen(url_or_request)
[ytdl_hook]   File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1998, in urlopen
[ytdl_hook]     return self._opener.open(req, timeout=self._socket_timeout)
[ytdl_hook]   File "/usr/lib/python2.7/urllib2.py", line 435, in open
[ytdl_hook]     response = meth(req, response)
[ytdl_hook]   File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
[ytdl_hook]     'http', request, response, code, msg, hdrs)
[ytdl_hook]   File "/usr/lib/python2.7/urllib2.py", line 473, in error
[ytdl_hook]     return self._call_chain(*args)
[ytdl_hook]   File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
[ytdl_hook]     result = func(*args)
[ytdl_hook]   File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
[ytdl_hook]     raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
[ytdl_hook] 
[ytdl_hook] youtube-dl failed, trying to play URL directly ... 
Failed to recognize file format.


Exiting... (Errors when loading file)

[fgaaghf]

Twitch now requires "client_id" for API calls. It was announced months ago, but turned on just 2 hours ago. Should be easy to fix.
See:
https://discuss.dev.twitch.tv/t/client-id-requirement-faqs/6108
https://blog.twitch.tv/client-id-required-for-kraken-api-calls-afbb8e95f843

[kasper93]

I applied quick fix locally. Also noticed that headers were not passed through so removed not needed ones (question is if Twitch-Api-Token is needed, it was working without it for some time). Youtube-dl should have it's own client-id probably instead of using web player one. Tho it is not for me to decide.

If someone wants quick fix until it is fixed in trunk:

diff --git a/youtube_dl/extractor/twitch.py b/youtube_dl/extractor/twitch.py
index 359a885..f393ab1 100644
--- a/youtube_dl/extractor/twitch.py
+++ b/youtube_dl/extractor/twitch.py
@@ -45,14 +45,13 @@ class TwitchBaseIE(InfoExtractor):

     def _call_api(self, path, item_id, note):
         headers = {
-            'Referer': 'http://api.twitch.tv/crossdomain/receiver.html?v=2',
-            'X-Requested-With': 'XMLHttpRequest',
+            'Client-ID': 'jzkbprff40iqj646a697cyrvl0zt2m6',
         }
         for cookie in self._downloader.cookiejar:
             if cookie.name == 'api_token':
                 headers['Twitch-Api-Token'] = cookie.value
         response = self._download_json(
-            '%s/%s' % (self._API_BASE, path), item_id, note)
+            '%s/%s' % (self._API_BASE, path), item_id, note, headers=headers)
         self._handle_error(response)
         return response

[Meneth32]

I tried kasper93's fix and it worked well. Thanks! :)

[weedy]

Who's client is that ID from?

[Hrxn]

His own account, I guess..

[leef627]

But I got this：
File "/usr/local/bin/youtube-dl", line 7, in
from youtube_dl import main
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/init.py", line 42, in
from .extractor import gen_extractors, list_extractors
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/init.py", line 9, in
from .extractors import *
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/extractors.py", line 951, in
from .twitch import (
File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/twitch.py", line 54
/%s' % (self._API_BASE, path), item_id, note, headers = headers)
^
SyntaxError: invalid syntax

[Volle2522]

None than you programmers get this shit. Twitch Ruined the whole Nvidia streaming ability... i really hope Nvidia Sue the shit out of you for doing your amazing decision.

[mralext20]

Er... What

[Hrxn]

Not sure, probably talking about the issues with the newly released GeForce Experience 3.0 software? There have been some problems, or so I heard...

Relevant suggestion for youtube-dl core:

Twitch is not the first site to use some form of authentication for API access, and it definitely won't be the last. It would probably be a good idea to extend the user authentication functionality of youtube-dl (--username, --password and support for .netrc, -n) with the ability to provide a user-specific client key/client ID/client token etc. pp. along the lines of the current way of providing account credentials.
Like, for example, --api-key or something like that to save to the .netrc file so every site API can be accessed in an very easy, straightforward and transparent way.

Because otherwise, what is happening now, we have an API key/Client ID hard-coded into the extractor file (<site>.py), and every user of youtube-dl will use this same key, every time and all the time they use the program. At some point, the API may limit the rate or bandwidth. Or worse.

[yan12125]

The problem of API keys is: they are often limited w.r.t mechanisms used in web pages. For example, some API keys have usage limitations, and some limits functionality. If there's a salient advantage of API keys, it's a good idea.

[Hrxn]

BTW, the Twitch extractor already supports reading from _NETRC_MACHINE

Not sure, but maybe it is possible to extend https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L665-L683, so that a new value/field can be retrieved from .netc, because right now we would only need those one value for CLIENT_ID, as it seems.

To be a bit more specific:

I don't have any experience with Python, but

[..]
 try:
                info = netrc.netrc().authenticators(netrc_machine)
                if info is not None:
                    username = info[0]
                    password = info[2]
                else:
[..]

info is read from the .netrc file, matching the correct machine entry, and to me it seems to be a normal array, so it should be possible to get and set another value here, right?

[kasper93]

Why do you want to override client id anyway? It is client/application identifier and is the same for all user on given application. Right now youtube-dl uses web player id. This value is not dependent on user or anything. It is not something that user should be able to change really.

[Hrxn]

This value is not dependent on user or anything.

Well, if that is the case and Client ID is not bound to a specific user account, it probably shouldn't be changed and my point becomes unnecessary. I don't know about the Twitch API, never used it so far, but other site's APIs use unique identifiers which are linked to an user account.

[canaryinadigitalmine]

Hi everyone! Kasper, me being a complete ignoramus in programming, do you think I would be able to apply your fix to my shadowplay software? If so, please tell me how this is done. Do I just copy paste that code into some file or something like that? Thank you very much! :D

[daemonfire300]

Windows 10 (newest build at this time)

> youtube-dl --version
2017.01.05

> youtube-dl https://www.twitch.tv/shenryyr/v/112981802 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.twitch.tv/shenryyr/v/112981802', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.01.05
[debug] Python version 3.5.2 - Windows-10-10.0.14393-SP0
[debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
[debug] Proxy map: {}
[twitch:vod] 112981802: Downloading vod info JSON
[twitch:vod] 112981802: Downloading vod access token
[twitch:vod] 112981802: Downloading m3u8 information
[debug] Invoking downloader on 'https://vod-secure.twitch.tv/8173b81bc18c0432da24_shenryyr_24167880736_579163048/chunked/index-dvr.m3u8'
[hlsnative] Downloading m3u8 manifest
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\site-packages\youtube_dl\downloader\common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\site-packages\youtube_dl\downloader\hls.py", line 63, in real_download
    manifest = self.ydl.urlopen(self._prepare_url(info_dict, man_url)).read()
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 472, in open
    response = meth(req, response)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 510, in error
    return self._call_chain(*args)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 444, in _call_chain
    result = func(*args)
  File "c:\users\fait-pc-new\appdata\local\programs\python\python35-32\lib\urllib\request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

[dstftw]

@daemonfire300 does not play in browser either.

[daemonfire300]

@dstftw my fault, I should have tried different VODs, seems like this one is just broken.