Unable to download webpage

[644]

I haven't encountered this before and I've tried restarting my system, but I am unable to get any further past "downloading webpage"

derplex@whisky:~# time youtube-dl -v https://www.youtube.com/watch?v=WbLEYQo98t0
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=WbLEYQo98t0']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.08
[debug] Python version 2.7.10 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-debian-7.11
[debug] exe versions: ffmpeg 2.7.2, ffprobe 2.7.2
[debug] Proxy map: {}
[youtube] WbLEYQo98t0: Downloading webpage
^C
ERROR: Interrupted by user

real    1m9.863s
user    0m0.956s
sys     0m0.048s

I did receive this message beforehand after ~ only 5 seconds and I'm unable to reproduce it with the -v flag

derplex@whisky:~# youtube-dl -F https://www.youtube.com/watch?v=WbLEYQo98t0
[youtube] WbLEYQo98t0: Downloading webpage
ERROR: Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)> (caused by URLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:590)'),))

[644]

I managed to reproduce the error with the -v flag after waiting a little longer

derplex@whisky:~# youtube-dl -v https://www.youtube.com/watch?v=WbLEYQo98t0
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=WbLEYQo98t0   ']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.08
[debug] Python version 2.7.10 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-debia   n-7.11
[debug] exe versions: ffmpeg 2.7.2, ffprobe 2.7.2
[debug] Proxy map: {}
[youtube] WbLEYQo98t0: Downloading webpage
ERROR: Unable to download webpage: <urlopen error EOF occurred in violation of p   rotocol (_ssl.c:590)> (caused by URLError(SSLEOFError(8, u'EOF occurred in viol    tion of protocol (_ssl.c:590)'),))
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 394, in    _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1998, in urlop    n
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/root/.pyenv/versions/2.7.10/lib/python2.7/urllib2.py", line 431, in op    n
    response = self._open(req, data)
  File "/root/.pyenv/versions/2.7.10/lib/python2.7/urllib2.py", line 449, in _o    en
    '_open', req)
  File "/root/.pyenv/versions/2.7.10/lib/python2.7/urllib2.py", line 409, in _c    ll_chain
    result = func(*args)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1004, in https_ope
    req, **kwargs)
  File "/root/.pyenv/versions/2.7.10/lib/python2.7/urllib2.py", line 1197, in d    _open
    raise URLError(err)

[yan12125]

FYI: #4578 (comment) may help.

[644]

I tried installing with the edited utils.py and it's producing the same result.

[yan12125]

Could you try Python 3?

[644]

This is the output when using python3 with the edited utils.py. Not sure why this is happening.

derplex@whisky~/youtube-dl/bin# python3 youtube-dl
Traceback (most recent call last):
  File "youtube-dl", line 3, in <module>
    import youtube_dl
ImportError: No module named youtube_dl

[yan12125]

Seems your version is coupled with Python 2 tightly and can't be used with Python 3. Could you try to download https://yt-dl.org/downloads/latest/youtube-dl and run it via python3 ./youtube-dl?

[644]

Same result. I'm going to try a different system.

derplex@whisky:~/ytdl# time python3 youtube-dl -v https://www.youtube.com/watch?v=WbLEYQo98t0
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=WbLEYQo98t0']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.08
[debug] Python version 3.2.3 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-Debian-7
[debug] exe versions: ffmpeg 2.7.2, ffprobe 2.7.2
[debug] Proxy map: {}
[youtube] WbLEYQo98t0: Downloading webpage
^C
ERROR: Interrupted by user

real    0m52.571s
user    0m0.924s
sys     0m0.040s

[yan12125]

If you want to try something more on the original system, usually the following command can be used to debug hang programs:

strace -f python3 ./youtube-dl -v https://www.youtube.com/watch?v=WbLEYQo98t0 2>& output.txt

output.txt would contain some debugging information.

[644]

There's issues with ipv6 but I get the same hang with the -4 flag.
http://91.121.76.10/output.txt

[yan12125]

Your device hangs when trying to connect to 172.217.20.46. This IP looks a valid IP for www.youtube.com. Does ping 172.217.20.46 give you any response?

[644]

Perhaps YouTube have blocked my IP?

derplex@whisky~l# ping 172.217.20.46
PING 172.217.20.46 (172.217.20.46) 56(84) bytes of data.
64 bytes from 172.217.20.46: icmp_req=22 ttl=58 time=4.44 ms
^C
--- 172.217.20.46 ping statistics ---
33 packets transmitted, 1 received, 96% packet loss, time 32246ms
rtt min/avg/max/mdev = 4.445/4.445/4.445/0.000 ms

[yan12125]

It's possible. Can you visit www.youtube.com in browsers?

[644]

It looks like they blocked me since I can connect to everything else except YouTube using my server as a proxy. It's a static IP so there's no changing it unless I get a new host. Any suggestion?

[yan12125]

How about changing to another DNS? Maybe not all YouTube IPs are blocking you.

[644]

How could I do that?

[yan12125]

A possible way is to change /etc/resolv.conf. Check Linux manuals.

I'm closing it first as it's not a youtube-dl issue. More discussions are welcome.