macOS 10.12 - urlopen error EOF occurred in violation of protocol (_ssl.c:590) #10782

grahammiln · 2016-09-27T20:06:29Z

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.09.27. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

I've verified and I assure that I'm running youtube-dl 2016.09.27

$ youtube-dl --version
2016.09.27

Before submitting an issue make sure you have:

At least skimmed through README and most notably FAQ and BUGS sections
Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

Bug report (encountered problems with youtube-dl)
Site support request (request for adding support for a new site)
Feature request (request for a new functionality)
Question
Other

Running the same command twice frequently results in differing numbers of pages being downloaded and parsed. It is rare to successfully fetch more than ten pages before an error or timeout occurs.

Run 1 of 2

$ youtube-dl -v -u "PRIVATE" -p 'PRIVATE' --format mp4 --download-archive youtube-dl-archive.txt --youtube-skip-dash-manifest --dateafter now-30day --ignore-errors --recode-video mp4 --output "%(upload_date)s-%(uploader)s-%(title)s.%(ext)s" -f best :ytsubs
[debug] System config: []
[debug] User config: [u'--format', u'mp4']
[debug] Command-line args: [u'-v', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'--format', u'mp4', u'--download-archive', u'youtube-dl-archive.txt', u'--youtube-skip-dash-manifest', u'--dateafter', u'now-30day', u'--ignore-errors', u'--recode-video', u'mp4', u'--output', u'%(upload_date)s-%(uploader)s-%(title)s.%(ext)s', u'-f', u'best', u':ytsubs']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.27
[debug] Python version 2.7.12 - Darwin-16.0.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.3, ffprobe 3.1.3, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:subscriptions] Downloading login page
[youtube:subscriptions] Logging in
WARNING: unable to log in: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>
[youtube:subscriptions] Youtube Subscriptions: Downloading webpage
[youtube:subscriptions] Youtube Subscriptions: Downloading page #1
[youtube:subscriptions] Youtube Subscriptions: Downloading page #2
[youtube:subscriptions] Youtube Subscriptions: Downloading page #3
[youtube:subscriptions] Youtube Subscriptions: Downloading page #4
[youtube:subscriptions] Youtube Subscriptions: Downloading page #5
ERROR: Unable to download JSON metadata: ('The read operation timed out',) (caused by SSLError('The read operation timed out',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 402, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 899, in http_response
    content = resp.read()
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 355, in read
    data = self._sock.recv(rbufsize)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 588, in read
    return self._read_chunked(amt)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 648, in _read_chunked
    value.append(self._safe_read(amt))
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 703, in _safe_read
    chunk = self.fp.read(min(amt, MAXAMOUNT))
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 756, in recv
    return self.read(buflen)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 643, in read
    v = self._sslobj.read(len)

Run 2 of 2

$ youtube-dl -v -u "PRIVATE" -p 'PRIVATE' --format mp4 --download-archive youtube-dl-archive.txt --youtube-skip-dash-manifest --dateafter now-30day --ignore-errors --recode-video mp4 --output "%(upload_date)s-%(uploader)s-%(title)s.%(ext)s" -f best :ytsubs
[debug] System config: []
[debug] User config: [u'--format', u'mp4']
[debug] Command-line args: [u'-v', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'--format', u'mp4', u'--download-archive', u'youtube-dl-archive.txt', u'--youtube-skip-dash-manifest', u'--dateafter', u'now-30day', u'--ignore-errors', u'--recode-video', u'mp4', u'--output', u'%(upload_date)s-%(uploader)s-%(title)s.%(ext)s', u'-f', u'best', u':ytsubs']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.27
[debug] Python version 2.7.12 - Darwin-16.0.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.3, ffprobe 3.1.3, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:subscriptions] Downloading login page
[youtube:subscriptions] Logging in
WARNING: unable to log in: <urlopen error ('_ssl.c:574: The handshake operation timed out',)>
[youtube:subscriptions] Youtube Subscriptions: Downloading webpage
[youtube:subscriptions] Youtube Subscriptions: Downloading page #1
[youtube:subscriptions] Youtube Subscriptions: Downloading page #2
ERROR: Unable to download JSON metadata: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)> (caused by URLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:590)'),))
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 402, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1011, in https_open
    req, **kwargs)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1198, in do_open
    raise URLError(err)

Description of your issue, suggested solution and other information

Downloading YouTube subscriptions fail on macOS 10.12 using a homebrew installed youtube-dl. The error appears to be related to openssl and maybe related to unsupported SSL/TLS protocols.

WARNING: unable to log in: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>

ERROR: Unable to download JSON metadata: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)> (caused by URLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:590)'),))
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 402, in _request_webpage

The computer has the latest openssl installed via brew and it appears to be available to python:

$ python -c "import ssl; print ssl.OPENSSL_VERSION"
OpenSSL 1.0.2j  26 Sep 2016

This error occurs on macOS 10.12 but not on the same computer running a virtual machine installed with macOS 10.11.

I suspect something within the recently released macOS 10.12 is the underlying cause.

yan12125 · 2016-09-28T06:38:04Z

Downloading :ytsubs works for me on Sierra with both the built-in Python and the one from Homebrew. Could you try:

Reinstall python-related packages in homebrew
Use macOS's built-in Python

grahammiln · 2016-09-28T08:26:26Z

Good to know it is not universal. There must be something misconfigured on my Mac.

Clearing out /Library/Python makes no difference.
Using macOS's built-in Python shows the same behaviour.
Running from a new user account shows the same behaviour; must be computer wide and not user specific.
Running from VMware Fusion/macOS 10.11 works; not related to hardware or Internet provider.

I will keep investigating.

yan12125 · 2016-09-28T08:46:21Z

FWIW, @terefang gave a workaround for similar issues at #4578 (comment). They're on Ubuntu, though.

grahammiln · 2016-09-28T09:30:09Z

Switching the SSL/TLS protocol to TLSv1_2 in utils.py:641 and using python3…

context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) #ssl.create_default_context(ssl.Purpose.SERVER_AUTH)

…appears to get further but the error returns after a few pages are downloaded.

I have also attempted the fix in the original StackOverflow answer but downgrading the protocol makes no difference:

$ python -m youtube_dl -u PRIVATE -p PRIVATE --format mp4 --download-archive youtube-dl-archive.txt --youtube-skip-dash-manifest --dateafter now-30day --ignore-errors --recode-video mp4 --output "%(upload_date)s-%(uploader)s-%(title)s.%(ext)s" -f best :ytsubs
[youtube:subscriptions] Downloading login page
[youtube:subscriptions] Logging in
WARNING: unable to log in: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>
[youtube:subscriptions] Youtube Subscriptions: Downloading webpage
[youtube:subscriptions] Youtube Subscriptions: Downloading page #1
ERROR: Unable to download JSON metadata: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)> (caused by URLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:590)'),))

As an aside, this behaviour began after upgrading from macOS 10.11 to 10.12, Sierra.

yan12125 · 2016-09-28T09:50:59Z

One more thing: YouTube use different servers for different regions. Could you paste the IP of accounts.google.com on your side?

nslookup accounts.google.com

-4 may help, too. This option forces IPv4.

grahammiln · 2016-09-28T11:16:30Z

I am in France, using the Internet service provider free.fr:

$ nslookup accounts.google.com
Server:     192.168.1.254
Address:    192.168.1.254#53

Non-authoritative answer:
Name:   accounts.google.com
Address: 172.217.18.237

grahammiln · 2016-09-28T12:12:42Z

This appears to be an intermittent problem. youtube-dl is now completing successfully; despite nothing being changed.

Thank you for your time and help.

I will re-open this report, if the problem reappears or I find a robust solution.

junmuzi · 2017-07-06T15:29:22Z

I still hit this issue on my ubuntu 16.04, could anybody help to solve my issue, thanks very much.

As follows show:
root@yingzi:~# proxychains youtube-dl https://www.youtube.com/watch?v=BcWX6lyRxsE
ProxyChains-3.1 (http://proxychains.sf.net)
[youtube] BcWX6lyRxsE: Downloading webpage
|D-chain|-<>-127.0.0.1:1080-<><>-61.91.161.217:443-<><>-OK
[youtube] BcWX6lyRxsE: Downloading video info webpage
|D-chain|-<>-127.0.0.1:1080-<><>-61.91.161.217:443-<><>-OK
[youtube] BcWX6lyRxsE: Extracting video information
[youtube] BcWX6lyRxsE: Downloading MPD manifest
|D-chain|-<>-127.0.0.1:1080-<><>-216.58.200.33:443-<><>-OK
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
|D-chain|-<>-127.0.0.1:1080-<><>-219.76.4.73:443-<><>-OK
ERROR: unable to download video data: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>

yan12125 · 2017-07-06T15:50:25Z

@junmuzi: There were reports that proxychains sometimes break youtube-dl. Seems you use SOCKS5? Just use the built-in socks5 support from youtube-dl:

youtube-dl --proxy socks5://127.0.0.1:1080 https://www.youtube.com/watch?v=BcWX6lyRxsE

junmuzi · 2017-07-06T16:09:32Z

@yan12125: Thank you very much. I have re-tried, but it will stop, seems youtube-dl can not support socks5 agent.

root@yingzi:~# youtube-dl --proxy socks5://127.0.0.1:1080 https://www.youtube.com/watch?v=BcWX6lyRxsE
[youtube] BcWX6lyRxsE: Downloading webpage

Somebody tell me it is due to the limit of youtube site. I don't know. :-)

junmuzi · 2017-07-06T16:15:13Z

@yan12125 :
Youtube-dl support SOCKS proxies since 2016.05.10 #84 (Ref: #402)

But my youtube-dl version:
root@yingzi:~# youtube-dl --version
2016.02.22

yan12125 · 2017-07-06T16:19:23Z

Then you need update it :) The latest version is 2017.07.02. The FAQ entry on update can help.

junmuzi · 2017-07-06T17:02:05Z

@yan12125:
Thank you very much. Solved my issue. It caused by proxychains.
root@yingzi:~# /root/anaconda2/bin/youtube-dl --proxy socks5://127.0.0.1:1080 -f best -f mp4 "https://www.youtube.com/watch?v=RTnNxbG2V5o"
[youtube] RTnNxbG2V5o: Downloading webpage
[youtube] RTnNxbG2V5o: Downloading video info webpage
[youtube] RTnNxbG2V5o: Extracting video information
[youtube] RTnNxbG2V5o: Downloading MPD manifest
[youtube] RTnNxbG2V5o: Downloading MPD manifest
[download] Destination: Mablethorpe DUNES Seaside Fun Fair Rides - Dodgems - On Ride POV-RTnNxbG2V5o.mp4
[download] 100% of 18.03MiB in 00:05

yan12125 added the cant-reproduce label Sep 28, 2016

grahammiln closed this Sep 28, 2016

yan12125 mentioned this issue Jul 6, 2017

ERROR: unable to download video data: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)> #13581

Closed

ytdl-org / youtube-dl

macOS 10.12 - urlopen error EOF occurred in violation of protocol (_ssl.c:590) #10782

macOS 10.12 - urlopen error EOF occurred in violation of protocol (_ssl.c:590) #10782

grahammiln commented Sep 27, 2016

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016 •

edited

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016

grahammiln commented Sep 28, 2016

junmuzi commented Jul 6, 2017 •

edited

yan12125 commented Jul 6, 2017

junmuzi commented Jul 6, 2017 •

edited

junmuzi commented Jul 6, 2017 •

edited

yan12125 commented Jul 6, 2017

junmuzi commented Jul 6, 2017 •

edited

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

macOS 10.12 - urlopen error EOF occurred in violation of protocol (_ssl.c:590) #10782

macOS 10.12 - urlopen error EOF occurred in violation of protocol (_ssl.c:590) #10782

Comments

grahammiln commented Sep 27, 2016

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.09.27. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

Run 1 of 2

Run 2 of 2

Description of your issue, suggested solution and other information

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016 • edited

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016

yan12125 commented Sep 28, 2016

grahammiln commented Sep 28, 2016

grahammiln commented Sep 28, 2016

junmuzi commented Jul 6, 2017 • edited

yan12125 commented Jul 6, 2017

junmuzi commented Jul 6, 2017 • edited

junmuzi commented Jul 6, 2017 • edited

yan12125 commented Jul 6, 2017

junmuzi commented Jul 6, 2017 • edited

Essential cookies

Always active

Analytics cookies

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.09.27. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

grahammiln commented Sep 28, 2016 •

edited

junmuzi commented Jul 6, 2017 •

edited

junmuzi commented Jul 6, 2017 •

edited

junmuzi commented Jul 6, 2017 •

edited

junmuzi commented Jul 6, 2017 •

edited