"YouTube said: Unable to extract video data" on some videos

[deepesh-agarwal]

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2020.07.28
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

youtube-dl -v -F 6sKluPf9-dk
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'6sKluPf9-dk']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 2.7.16 (CPython) - Linux-4.19.0-6-amd64-x86_64-with-debian-10.2
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[youtube] 6sKluPf9-dk: Downloading webpage
ERROR: 6sKluPf9-dk: YouTube said: Unable to extract video data
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 1893, in _real_extract
    'YouTube said: %s' % unavailable_message, expected=True, video_id=video_id)
ExtractorError: 6sKluPf9-dk: YouTube said: Unable to extract video data

Description

Most videos work fine but few videos like the one above showing error. ExtractorError: 6sKluPf9-dk: YouTube said: Unable to extract video data

[llacb47]

#26152

[deepesh-agarwal]

#26100 Fixed most of the videos for me but the one as shown below is still not working.

youtube-dl -F -v 7takIh1nK0s
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-F', u'-v', u'7takIh1nK0s']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 2.7.16 (CPython) - Linux-4.19.0-6-amd64-x86_64-with-debian-10.2
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[youtube] 7takIh1nK0s: Downloading webpage
ERROR: 7takIh1nK0s: YouTube said: Unable to extract video data
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 1893, in _real_extract
    'YouTube said: %s' % unavailable_message, expected=True, video_id=video_id)
ExtractorError: 7takIh1nK0s: YouTube said: Unable to extract video data

[omgitsmoe]

Video "7takIh1nK0s" fails with a different error for me (having applied the fix from #26100): youtube_dl.utils.ExtractorError: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.

So the age-gate detection works but the trick that is used to circumvent it doesn't, since it simulates watching an embedded video which obviously doesn't work when the video owner disables it. So for this video you will have to log into YouTube and pass that login cookie to youtube-dl (see How do I pass cookies to youtube-dl?).

[ddawson]

For me, youtube-dl has been working only on non–age-restricted videos and with no authentication cookies. If I provide other cookies, it's fine, but certain cookies are causing problems.

After quite a bit of experimentation, I've determined that the VISITOR_INFO1_LIVE cookie is causing problems. If absent, youtube-dl can download a video, even an age-restricted video if LOGIN_INFO is provided. But if VISITOR_INFO1_LIVE is provided, YouTube provides different output that is breaking the youtube extractor. If I use a cookie jar with LOGIN_INFO in it, this is a problem, since YouTube is providing VISITOR_INFO1_LIVE, and youtube-dl writes it in, so it can only download one video before it starts failing. Or none, in the case of a playlist.

The web page for age-restricted videos when not authenticated seems to be different as well.

In case it helps, I provide the web page outputs for an example age-restricted video when not authenticated, when authenticated but without VISITOR_INFO1_LIVE ("first time"), and authenticated with that cookie ("second time").

It would be helpful to suppress that cookie until someone figures out how to parse the current format.

[omgitsmoe]

@ddawson have you applied the fix in #26100?

The video you provided works fine for me using the fix in #26100 even when i pass a cookie file containing VISITOR_INFO1_LIVE.

[ddawson]

That fix did indeed fix age gate detection, so that's good. It needs to be merged.

What it did not fix, however, is the case of being authenticated as I detailed above. That still fails after the first time.

Here is a patch to suppress that cookie. I'm not submitting a PR because I'm not sure this is the best way to fix the problem. But it's at least a functional workaround. (EDIT: actually, no, it needs to be applied to other downloads besides just a video watch page.)

Here is a patch that should cover all downloads.

[ddawson]

Should this issue really be closed? Commit 16ee69c fixes the age gate detection, but it does not solve the problem I have been describing above. If I use an authentication cookie, the "Unable to extract video data" error still shows up whenever I have VISITOR_INFO1_LIVE (i.e. after once downloading anything). Not using cookies—and thus not being signed in—works fine in general. However, I am concerned about a couple of issues:

• Private videos are only accessible to the uploader and those other users they specifically authorize, which requires being signed in. Thus, currently, youtube-dl has trouble downloading private videos.
• If I understand correctly, youtube-dl currently works around age gating by simulating an embedded player, where YouTube doesn't check authentication. I wonder, how long until they close this loophole? It may be helpful to properly handle authentication sooner rather than later.