Twitter "age restricted" videos

[marcespie]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• I'm reporting that a supported site is broken
• I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

France

Provide a description that is worded well enough to be understood

Some twitter videos are not available. I believe it has to do with age-restricted content.

I"ve selected something reasonably tame as an example (nude breasts bondage, skip if that offends you):

https://twitter.com/i/status/1650552998513324037

my account which can see those videos has been added to netrc.

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• If using API, add 'verbose': True to YoutubeDL params instead
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', '--netrc', 'https://twitter.com/i/status/1650552998513324037']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d]
[debug] Python 3.10.10 (CPython amd64 64bit) - OpenBSD-7.3-amd64-64bit-ELF (LibreSSL 3.7.2, libc 97.0)
[debug] exe versions: ffmpeg 4.4.3 (setts), ffprobe 4.4.3
[debug] Optional libraries: brotli-1.0.9, certifi-2022.12.07, pycrypto-3.15.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[twitter] Extracting URL: https://twitter.com/i/status/1650552998513324037
[twitter] 1650552998513324037: Downloading guest token
[twitter] 1650552998513324037: Downloading JSON metadata
WARNING: [twitter] Twitter API gave 404 response, retrying with deprecated auth token. Only one media item can be extracted
[twitter] 1650552998513324037: Downloading guest token
[twitter] 1650552998513324037: Downloading JSON metadata
ERROR: [twitter] 1650552998513324037: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/twitter.py", line 974, in _real_extract
    status = traverse_obj(self._call_api(f'statuses/show/{twid}.json', twid, {
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/twitter.py", line 136, in _call_api
    raise last_error
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/twitter.py", line 130, in _call_api
    result = self._download_json(
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1048, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1012, in download_handle
    res = self._download_webpage_handle(
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 882, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 839, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 821, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/usr/local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3742, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

[pukkandan]

#6763

[marcespie]

Sorry for not finding the original one... you gotta admit the --cookies-from-browser stuff is not intuitive and should be added to the error message.

Anyway, thanks

[pukkandan]

should be added to the error message.

Agreed, but at the same time,

• I've checked that all provided URLs are playable in a browser with the same IP and same login details

[marcespie]

Heck, I didn't even know about --cookies-from-browser before this error report. I was still in the netrc stage.
When there are too many (arguably) options, you gotta provide some meaningful error messages to help the user (I know this is a complicated situation, I'm maintaining tools where error message can be and have been improved)

Or: more specifically, I can now download what I want from twitter, on a personal level. But it would still be great for other users to enjoy the same freedom without having to wade through 10+ pages of manpage.

[bashonly]

you gotta provide some meaningful error messages to help the user

this is the plan, as described in the other issue

meaningful error message is not given currently because twitter site changes broke our error handling