Download clips twitch broken

[pwd03653]

command:
youtube-dl -v https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.twitch.tv/f1reaoe/clip/CautiousEn
ticingSardinePJSugar']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-8.1-6.3.9600
[debug] exe versions: ffmpeg 4.0
[debug] Proxy map: {}
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by HTTPErr
or()); please report this issue on https://yt-dl.org/bug . Make sure you are usi
ng the latest version; type youtube-dl -U to update. Be sure to call youtube-d
l with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\YoutubeDL.py", line 2237, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_defau
lt

[remitamine]

youtube-dl https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
[download] Destination: F1Re Graphics-579811939.mp4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 7853k  100 7853k    0     0   356k      0  0:00:22  0:00:22 --:--:--  434k
[curl] Downloaded 8042261 bytes
[download] 100% of 7.67MiB in 00:22

[kuchenmitsahne]

I'm encountering the same issue whether I use the Windows executable with embedded Python (I'm on Windows 10 Pro) or the standard version installed via pip.

With the Windows executable:

youtube-dl.exe --verbose https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-94543-g8fcc5d963e, ffprobe N-94543-g8fcc5d963e, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\YoutubeDL.py", line 2237, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

With the pip version:

youtube-dl --verbose https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.7.2 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg N-94543-g8fcc5d963e, ffprobe N-94543-g8fcc5d963e, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "c:\python37\lib\site-packages\youtube_dl\extractor\common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "c:\python37\lib\site-packages\youtube_dl\YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "c:\python37\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "c:\python37\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "c:\python37\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "c:\python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "c:\python37\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Oddly, on my Android phone (running in Termux), I get a different error which seems to indicate that the extractor is checking the state of the livestream, which shouldn't be relevant for a clip. I think this is just an issue with the regular expressions used to sort the links into the different extractor models; clip URLs of the form https://www.twitch.tv/{channel_id}/clip/{video_id} are not rejected by the TwitchStreamIE regex, which simply captures the channel ID while ignoring the rest of the URL:

youtube-dl/youtube_dl/extractor/twitch.py

Lines 517 to 526 in cf80ff1

	class TwitchStreamIE(TwitchBaseIE):
	IE_NAME = 'twitch:stream'
	_VALID_URL = r'''(?x)
	https?://
	(?:
	(?:(?:www|go|m)\.)?twitch\.tv/|
	player\.twitch\.tv/\?.*?\bchannel=
	)
	(?P<id>[^/#?]+)
	'''

youtube-dl/youtube_dl/extractor/twitch.py

Lines 646 to 648 in cf80ff1

	class TwitchClipsIE(TwitchBaseIE):
	IE_NAME = 'twitch:clips'
	_VALID_URL = r'https?://(?:clips\.twitch\.tv/(?:embed\?.*?\bclip=|(?:[^/]+/)*)|(?:www\.)?twitch\.tv/[^/]+/clip/)(?P<id>[^/?#&]+)'

When I use an unambiguous clip URL in Termux, I get a similar HTTP error to those shown above.

In Termux (ambiguous regex?):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-config', '--verbose', 'https://twitch.tv/f1reaoe/CautiousEnticingSardinePJSugar']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.8.0 (CPython) - Linux-4.9.112-16352588-aarch64-with-libc
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[twitch:stream] f1reaoe: Downloading stream JSON
ERROR: f1reaoe is offline
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/youtube_dl/extractor/twitch.py", line 583, in _real_extract
    raise ExtractorError('%s is offline' % channel_id, expected=True)
youtube_dl.utils.ExtractorError: f1reaoe is offline

In Termux (unambiguous clip URL):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-config', '--verbose', 'https://clips.twitch.tv/CautiousEnticingSardinePJSugar']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.8.0 (CPython) - Linux-4.9.112-16352588-aarch64-with-libc
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/data/data/com.termux/files/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/data/data/com.termux/files/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/data/data/com.termux/files/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/data/data/com.termux/files/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/data/data/com.termux/files/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Taking a peek at the code, one thing I noticed is that the extractor makes a request using Twitch API v2, which has long since been deprecated.

youtube-dl/youtube_dl/extractor/twitch.py

Lines 676 to 681 in cf80ff1

	def _real_extract(self, url):
	video_id = self._match_id(url)
	status = self._download_json(
	'https://clips.twitch.tv/api/v2/clips/%s/status' % video_id,
	video_id)

In fact, when I try to make the requisite API call for the example clip at hand, I get a telling response:

curl -X GET https://clips.twitch.tv/api/v2/clips/CautiousEnticingSardinePJSugar/status
{"message": "This Clips route is deprecated."}

Crucially, I've been noticing exact same response with an entirely different application that also uses Twitch API v2 and makes the exact same API call: an Android app called ImgurViewer which supports viewing Twitch clips (link to the relevant line of code on BitBucket). The issue with ImgurViewer (which has rendered it unable to display Twitch clips) only started in the last few days (can't remember exactly when, but definitely no more than a week ago), and one strange thing I've noticed is that it'll still work occasionally, although only for short periods of time. This may explain why @remitamine was unable to reproduce the bug.

[broizter]

Same problem here. Behaves exactly the same on Linux and Windows.

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.twitch.tv/f1reaoe/clip/CautiousEnticingSardinePJSugar']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-93747-g6223d076e3
[debug] Proxy map: {}
[twitch:clips] CautiousEnticingSardinePJSugar: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\YoutubeDL.py", line 2237, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default