[Q] Not available videos become available as soon as I proxy through the same machine #18456

delivrance · 2018-12-08T17:52:55Z

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2018.12.03. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

I've verified and I assure that I'm running youtube-dl 2018.12.03

Before submitting an issue make sure you have:

At least skimmed through the README, most notably the FAQ and BUGS sections
Searched the bugtracker for similar issues including closed ones
Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

Bug report (encountered problems with youtube-dl)
Site support request (request for adding support for a new site)
Feature request (request for a new functionality)
Question
Other

Description of your issue, suggested solution and other information

I'm getting This video is not available errors for some videos lately using, for example:
youtube-dl -F <video_id>.

The weird thing is that the same video_ids work just fine if I use a proxy which is actually running on the same machine:
youtube-dl --proxy http://user:pass@localhost:1234 -F <video_id>

I tried with --rm-cache-dir, installed again, rebooted... Any other ideas?

dstftw · 2018-12-08T18:01:32Z

Anything from geo restriction to mis-copy-pasted video id.

delivrance · 2018-12-08T18:27:47Z

@dstftw That was kinda too fast.

Tested videos are not geo restricted, I can view them in the browser (as I ticked in the list).
The errors started to pop up just randomly today from my logs; I didn't mistype/mispasted any video id.
youtube-dl simply starts to work fine when I literally proxy it through the same machine (same IP address).

--verbose output doesn't seem to be particularly useful, but here you go:

1. Click to expand (without proxy)

[debug] System config: []
  [debug] User config: []
  [debug] Custom config: []
  [debug] Command-line args: ['-v', '-F', 'SpMCGXWzCS0']
  [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
  [debug] youtube-dl version 2018.12.03
  [debug] Python version 3.7.1 (CPython) - Linux-4.9.0-8-amd64-x86_64-with-debian-9.6
  [debug] exe versions: ffmpeg 4.1, ffprobe 4.1
  [debug] Proxy map: {}
  [youtube] SpMCGXWzCS0: Downloading webpage
  [youtube] SpMCGXWzCS0: Downloading video info webpage
  ERROR: This video is not available.
  Traceback (most recent call last):
    File "/usr/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 792, in extract_info
      ie_result = ie.extract(url)
    File "/usr/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 508, in extract
      ie_result = self._real_extract(url)
    File "/usr/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 1920, in _real_extract
      raise ExtractorError(error_message, expected=True)
  youtube_dl.utils.ExtractorError: This video is not available.

2. Click to expand (with proxy)

[debug] System config: []
  [debug] User config: []
  [debug] Custom config: []
  [debug] Command-line args: ['--proxy', 'http://squid:sucacazz@localhost:1234', '-v', '-F', 'SpMCGXWzCS0']
  [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
  [debug] youtube-dl version 2018.12.03
  [debug] Python version 3.7.1 (CPython) - Linux-4.9.0-8-amd64-x86_64-with-debian-9.6
  [debug] exe versions: ffmpeg 4.1, ffprobe 4.1
  [debug] Proxy map: {'http': 'http://squid:sucacazz@localhost:1234', 'https': 'http://squid:sucacazz@localhost:1234'}
  [youtube] SpMCGXWzCS0: Downloading webpage
  [youtube] SpMCGXWzCS0: Downloading video info webpage
  [youtube] {18} signature length 43.42, html5 player vflf5K4kk
  [youtube] {36} signature length 43.42, html5 player vflf5K4kk
  [youtube] {17} signature length 43.42, html5 player vflf5K4kk
  [youtube] {264} signature length 43.42, html5 player vflf5K4kk
  [youtube] {271} signature length 43.42, html5 player vflf5K4kk
  [youtube] {137} signature length 43.42, html5 player vflf5K4kk
  [youtube] {248} signature length 43.42, html5 player vflf5K4kk
  [youtube] {136} signature length 43.42, html5 player vflf5K4kk
  [youtube] {247} signature length 43.42, html5 player vflf5K4kk
  [youtube] {135} signature length 43.42, html5 player vflf5K4kk
  [youtube] {244} signature length 43.42, html5 player vflf5K4kk
  [youtube] {134} signature length 43.42, html5 player vflf5K4kk
  [youtube] {243} signature length 43.42, html5 player vflf5K4kk
  [youtube] {133} signature length 43.42, html5 player vflf5K4kk
  [youtube] {242} signature length 43.42, html5 player vflf5K4kk
  [youtube] {160} signature length 43.42, html5 player vflf5K4kk
  [youtube] {278} signature length 43.42, html5 player vflf5K4kk
  [youtube] {140} signature length 43.42, html5 player vflf5K4kk
  [youtube] {171} signature length 43.42, html5 player vflf5K4kk
  [youtube] {249} signature length 43.42, html5 player vflf5K4kk
  [youtube] {250} signature length 43.42, html5 player vflf5K4kk
  [youtube] {251} signature length 43.42, html5 player vflf5K4kk
  [info] Available formats for SpMCGXWzCS0:
  format code  extension  resolution note
  249          webm       audio only DASH audio   64k , opus @ 50k, 1.41MiB
  250          webm       audio only DASH audio   83k , opus @ 70k, 1.84MiB
  140          m4a        audio only DASH audio  128k , m4a_dash container, mp4a.40.2@128k, 3.07MiB
  171          webm       audio only DASH audio  143k , vorbis@128k, 3.27MiB
  251          webm       audio only DASH audio  161k , opus @160k, 3.65MiB
  160          mp4        256x144    144p   18k , avc1.4d400c, 25fps, video only, 324.37KiB
  133          mp4        426x240    240p   31k , avc1.4d4015, 25fps, video only, 497.66KiB
  278          webm       256x144    144p   34k , webm container, vp9, 25fps, video only, 569.47KiB
  134          mp4        640x360    360p   59k , avc1.4d401e, 25fps, video only, 913.90KiB
  242          webm       426x240    240p   60k , vp9, 25fps, video only, 885.83KiB
  135          mp4        854x480    480p   98k , avc1.4d401e, 25fps, video only, 1.41MiB
  243          webm       640x360    360p  103k , vp9, 25fps, video only, 1.48MiB
  244          webm       854x480    480p  153k , vp9, 25fps, video only, 2.15MiB
  136          mp4        1280x720   720p  164k , avc1.4d401f, 25fps, video only, 2.38MiB
  247          webm       1280x720   720p  251k , vp9, 25fps, video only, 3.53MiB
  137          mp4        1920x1080  1080p  274k , avc1.640028, 25fps, video only, 4.00MiB
  248          webm       1920x1080  1080p  411k , vp9, 25fps, video only, 5.78MiB
  264          mp4        2536x1426  1440p  442k , avc1.640032, 25fps, video only, 6.44MiB
  271          webm       2536x1426  1440p 1502k , vp9, 25fps, video only, 15.41MiB
  17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k, 987.50KiB
  36           3gp        320x180    small , mp4v.20.3, mp4a.40.2, 2.30MiB
  18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k, 6.44MiB (best)

What kind of other details do you need?

dstftw · 2018-12-08T18:30:17Z

Unmodified verbose logs with --dump-pages --print-traffic.

delivrance · 2018-12-08T19:13:51Z

@dstftw No proxy (non-working). With proxy (working)

dstftw · 2018-12-08T19:21:19Z

Does http://yt-dl.org/ip shows the same IPs?

delivrance · 2018-12-08T19:40:52Z

I am indeed getting different IPs. It turns out that connections through the proxy are done via IPv6 instead; youtube-dl -F -6 <video_id> workarounds my issue, but still I have no idea why only some video fail on IPv4. Probably youtube-side issues?

dstftw · 2018-12-08T19:48:55Z

Well, looking at first webpages from both runs it clearly seen that both cases are served with video-not-available webpage (video can't be watched in browser in both cases). But in case of IPv6 ytplayer.config is still included in webpage's source making it available to download by youtube-dl. So it looks like a Youtube's bug and a loophole for youtube-dl at the same time.

delivrance · 2018-12-08T20:05:36Z

Alright, thanks. Let's see if something changes in the next days, I'll eventually report back here.

What made me say I'm able to watch them in browser was an actual test I did by proxying my local machine through that server, and videos were playable.

One more little question while we are at it: my application is actually embedding youtube-dl in a python script, so I would need to force IPv6 in there, but I couldn't find anything related to IPv6 nor IPv4 here. I tried forceipv6, force-ipv6, 6, ipv6 and some other variations = True.

dstftw · 2018-12-08T20:13:25Z

It's source_address: :: for any v6, 0.0.0.0 for any v4 or concrete IPs.

delivrance · 2018-12-08T20:25:28Z

Ouch, that was pretty hidden. It worked anyway, thanks.

dstftw closed this Dec 8, 2018

dstftw added the incomplete label Dec 8, 2018

dstftw removed the incomplete label Dec 8, 2018

ytdl-org / youtube-dl

[Q] Not available videos become available as soon as I proxy through the same machine #18456

[Q] Not available videos become available as soon as I proxy through the same machine #18456

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018 •

edited

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

[Q] Not available videos become available as soon as I proxy through the same machine #18456

[Q] Not available videos become available as soon as I proxy through the same machine #18456

Comments

delivrance commented Dec 8, 2018

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.12.03. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

Description of your issue, suggested solution and other information

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018 • edited

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

dstftw commented Dec 8, 2018

delivrance commented Dec 8, 2018

Essential cookies

Always active

Analytics cookies

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2018.12.03. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

dstftw commented Dec 8, 2018 •

edited