Trying to download certain Youtube videos through tor produces "Video not available" error in some cases. Might this be addressed from the yt-dlp side? #7110

johnahnase · 2023-05-23T16:50:27Z

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

I'm asking a question and not reporting a bug or requesting a feature
I've looked through the README
I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood

Attempting to download certain videos on youtube through tor through Invidous .onion frontends persistently fails with the following message:

ERROR: [youtube] [whateverVideoID]: This video is not available

I have failed to download certain videos like this over the course of a couple of months or so. I've tested downloading some videos where this issue occurs without going through tor, and it has always worked.

So, I am wondering if this is a an issue that cannot be resolved as far as yt-dlp is concerned, or not.

Provide verbose output that clearly demonstrates the problem

Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
If using API, add 'verbose': True to YoutubeDL params instead
Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', '-f', 'ba', '-x', '--audio-format', 'flac', '--audio-quality', '0', '--prefer-free-formats', 'http://grwp24hodrefzvjjuccrkw3mjq4tzhaaq32amf33dzpmuxe7ilepcmad.onion/playlist?list=PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d]
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-6.2.6-76060206-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-11.0.2
[debug] Proxy map: {}
[debug] Loaded 1801 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[youtube:tab] Extracting URL: http://grwp24hodrefzvjjuccrkw3mjq4tzhaaq32amf33dzpmuxe7ilepcmad.onion/playlist?list=PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T
[youtube:tab] PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T: Downloading webpage
WARNING: [youtube:tab] Unable to download webpage: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: 'Too Many Requests'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[youtube:tab] PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T: Downloading API parameters API JSON
[youtube:tab] PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T: Downloading API JSON
WARNING: [youtube:tab] YouTube said: INFO - 14 unavailable videos are hidden
[youtube:tab] PLycVTiaj8OI9COuVDJdw_RdBWy11ALc4T: Redownloading playlist API JSON with unavailable videos
[download] Downloading playlist: 01-The Beatles - Please Please me (full album)
[youtube:tab] Playlist 01-The Beatles - Please Please me (full album): Downloading 14 items of 14
[download] Downloading item 1 of 14
[youtube] Extracting URL: https://www.youtube.com/watch?v=oxwAB3SECtc
[youtube] oxwAB3SECtc: Downloading webpage
WARNING: [youtube] Unable to download webpage: HTTP Error 429: Too Many Requests
[youtube] oxwAB3SECtc: Downloading android player API JSON
[youtube] oxwAB3SECtc: Downloading iframe API JS
WARNING: [youtube] Unable to download webpage: HTTP Error 429: Too Many Requests
[youtube] oxwAB3SECtc: Downloading web player API JSON
ERROR: [youtube] oxwAB3SECtc: This video is not available
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/youtube.py", line 4066, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1172, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

The text was updated successfully, but these errors were encountered:

gamer191 · 2023-05-24T07:33:57Z

Invidious URLs aren’t supported and are redirected to YouTube.
I am curious about the 429 errors though. Is YouTube blocked in your country or something?

I've tested downloading some videos where this issue occurs without going through tor, and it has always worked.

Are you sure? Do they work if you just use YouTube.com?

gamer191 · 2023-05-24T07:34:37Z

(Please post a verbose log regarding my 2nd question)

coletdjnz · 2023-05-24T08:20:47Z

The issue here is the content is not available in the country of your TOR exit IP, so nothing we can feasibly do on the yt-dlp side.

As gamer191 mentioned, invidious urls are translated to youtube.com urls.

As for the 429, The TOR exit IP you are using is blocked from youtube.com. This is common on TOR. But this shouldn't affect yt-dlp in this case as we have fallbacks which are being used here.

gamer191 · 2023-05-24T08:59:29Z

As for the 429, The TOR exit IP you are using is blocked from youtube.com. This is common on TOR. But this shouldn't affect yt-dlp in this case as we have fallbacks which are being used here.

Are you sure? I've definitely accessed Youtube from tor before. Perhaps it's fingerprinting tor users?

But this shouldn't affect yt-dlp in this case as we have fallbacks which are being used here.

Um, the fallbacks must be broken, since the log shows ERROR: [youtube] oxwAB3SECtc: This video is not available

coletdjnz · 2023-05-24T09:23:26Z

As for the 429, The TOR exit IP you are using is blocked from youtube.com. This is common on TOR. But this shouldn't affect yt-dlp in this case as we have fallbacks which are being used here.

Are you sure? I've definitely accessed Youtube from tor before. Perhaps it's fingerprinting tor users?

But this shouldn't affect yt-dlp in this case as we have fallbacks which are being used here.

Um, the fallbacks must be broken, since the log shows ERROR: [youtube] oxwAB3SECtc: This video is not available

I think you're getting mixed up between the 429 on the webpage (the IP is blocked from YouTube) and the video availability itself on YouTube (likely due to geo restrictions)

The 429s come and go and Tor has multiple exit IPs. It is irrelevant here since we use Innertuje API as a fallback to get the same data the webpage does when it fails.

johnahnase · 2023-05-24T13:40:29Z

Are you sure? Do they work if you just use YouTube.com?

$ yt-dlp -vU -f ba -x --audio-format flac --audio-quality 0 --prefer-free-formats https://www.youtube.com/watch?v=czw8eqepir8

success -->

 [debug] Command-line config: ['-vU', '-f', 'ba', '-x', '--audio-format', 'flac', '--audio-quality', '0', '--prefer-free-formats', 'https://www.youtube.com/watch?v=czw8eqepir8']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d]
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-6.2.6-76060206-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-11.0.2
[debug] Proxy map: {}
[debug] Loaded 1801 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[youtube] Extracting URL: https://www.youtube.com/watch?v=czw8eqepir8
[youtube] czw8eqepir8: Downloading webpage
[youtube] czw8eqepir8: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] czw8eqepir8: Downloading 1 format(s): 251
[debug] Invoking http downloader on "https://rr5---sn-uxaxovg-5go6.googlevideo.com/videoplayback?expire=1685009667&ei=o-BuZKrxLJzw7AT35ovICw&ip=94.234.100.251&id=o-APxoD6bAZ37sgT-sO3BxT9RuYYkvBDhvhhTPSaMTaApI&itag=251&source=youtube&requiressl=yes&mh=If&mm=31%2C29&mn=sn-uxaxovg-5go6%2Csn-5go7ynlk&ms=au%2Crdu&mv=m&mvi=5&pl=20&gcr=se&initcwndbps=2437500&spc=qEK7B5zjTOBroz0E1V_S-pUqohUmP3A&vprv=1&svpuc=1&mime=audio%2Fwebm&gir=yes&clen=2049062&dur=120.861&lmt=1575214211426258&mt=1684987777&fvip=1&keepalive=yes&fexp=24007246%2C51000023&c=ANDROID&txp=5431432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cgcr%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgaOkKOElpg1Xhj_f9W6vFdB60N5N4YQ4qWd0_ytbm1Y0CIQDqP5COlIwVwYnGltIi6LbYv7XHIaNiL2VAas1HJKMdBQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgLEUU_0g3B3_amw9wjRfglcPi0sThsheHd2xnhLsNyfUCIQDvdqjyro1h2ACQE88DJ9CQjM9W49iEmgbVijXE2R25BQ%3D%3D"
[download] Destination: Please Please Me (Remastered 2009) [czw8eqepir8].webm
[download] 100% of    1.95MiB in 00:00:00 at 3.20MiB/s
[debug] ffmpeg command line: ffprobe -show_streams 'file:Please Please Me (Remastered 2009) [czw8eqepir8].webm'
[ExtractAudio] Destination: Please Please Me (Remastered 2009) [czw8eqepir8].flac
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Please Please Me (Remastered 2009) [czw8eqepir8].webm' -vn -acodec flac -movflags +faststart 'file:Please Please Me (Remastered 2009) [czw8eqepir8].flac'
Deleting original file Please Please Me (Remastered 2009) [czw8eqepir8].webm (pass -k to keep)

|

$ torsocks yt-dlp -vU -f ba -x --audio-format flac --audio-quality 0 --prefer-free-formats http://grwp24hodrefzvjjuccrkw3mjq4tzhaaq32amf33dzpmuxe7ilepcmad.onion/watch?v=czw8eqepir8

failure -->

[debug] Command-line config: ['-vU', '-f', 'ba', '-x', '--audio-format', 'flac', '--audio-quality', '0', '--prefer-free-formats', 'http://grwp24hodrefzvjjuccrkw3mjq4tzhaaq32amf33dzpmuxe7ilepcmad.onion/watch?v=czw8eqepir8']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d]
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-6.2.6-76060206-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-11.0.2
[debug] Proxy map: {}
[debug] Loaded 1801 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[youtube] Extracting URL: http://grwp24hodrefzvjjuccrkw3mjq4tzhaaq32amf33dzpmuxe7ilepcmad.onion/watch?v=czw8eqepir8
[youtube] czw8eqepir8: Downloading webpage
[youtube] czw8eqepir8: Downloading android player API JSON
ERROR: [youtube] czw8eqepir8: Video unavailable. This video is not available
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/youtube.py", line 4066, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/home/mruser/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1172, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

|

$ torsocks yt-dlp -vU -f ba -x --audio-format flac --audio-quality 0 --prefer-free-formats https://www.youtube.com/watch?v=czw8eqepir8

This last one fails with the same output as the above failure, except for the URL being of Youtube rather than Invidious, obviously.

gamer191 · 2023-05-24T14:10:38Z

@johnahnase um, both logs demonstrate identical failures

gamer191 · 2023-05-24T14:12:53Z

I think you're getting mixed up between the 429 on the webpage (the IP is blocked from YouTube) and the video availability itself on YouTube (likely due to geo restrictions)

right, I didn't realise that Video unavailable meant the video was geo-blocked. Is that the expected/known behaviour?

johnahnase · 2023-05-25T04:17:08Z

@johnahnase um, both logs demonstrate identical failures

Whoops. Fixed.

pukkandan · 2023-06-16T05:23:53Z

$ yt-dlp -vU -f ba -x --audio-format flac --audio-quality 0 --prefer-free-formats https://www.youtube.com/watch?v=czw8eqepir8

*success -->

Is this without tor?

johnahnase · 2023-06-19T15:34:37Z

Is this without tor?

Yes. I only use tor by including 'torsocks' at the start of the command.

pukkandan · 2023-06-20T02:13:09Z

Then it's the tor endpoint being blocked by youtube. We can't actually extract from the onion site, only forwards it to youtube.com

johnahnase added the question Question label May 23, 2023

coletdjnz closed this as completed May 24, 2023

coletdjnz reopened this May 24, 2023

coletdjnz added the external issue Issue with an external tool label May 24, 2023

johnahnase closed this as completed Jun 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trying to download certain Youtube videos through tor produces "Video not available" error in some cases. Might this be addressed from the yt-dlp side? #7110

Trying to download certain Youtube videos through tor produces "Video not available" error in some cases. Might this be addressed from the yt-dlp side? #7110

johnahnase commented May 23, 2023

gamer191 commented May 24, 2023

gamer191 commented May 24, 2023

coletdjnz commented May 24, 2023 •

edited

gamer191 commented May 24, 2023

coletdjnz commented May 24, 2023

johnahnase commented May 24, 2023 •

edited

gamer191 commented May 24, 2023

gamer191 commented May 24, 2023 •

edited

johnahnase commented May 25, 2023

pukkandan commented Jun 16, 2023

johnahnase commented Jun 19, 2023

pukkandan commented Jun 20, 2023

Trying to download certain Youtube videos through tor produces "Video not available" error in some cases. Might this be addressed from the yt-dlp side? #7110

Trying to download certain Youtube videos through tor produces "Video not available" error in some cases. Might this be addressed from the yt-dlp side? #7110

Comments

johnahnase commented May 23, 2023

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Please make sure the question is worded well enough to be understood

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

gamer191 commented May 24, 2023

gamer191 commented May 24, 2023

coletdjnz commented May 24, 2023 • edited

gamer191 commented May 24, 2023

coletdjnz commented May 24, 2023

johnahnase commented May 24, 2023 • edited

gamer191 commented May 24, 2023

gamer191 commented May 24, 2023 • edited

johnahnase commented May 25, 2023

pukkandan commented Jun 16, 2023

johnahnase commented Jun 19, 2023

pukkandan commented Jun 20, 2023

coletdjnz commented May 24, 2023 •

edited

johnahnase commented May 24, 2023 •

edited

gamer191 commented May 24, 2023 •

edited