Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SSL error on running fetch_housing_data() #46

Closed
aishniparab opened this issue Jun 26, 2017 · 16 comments
Closed

SSL error on running fetch_housing_data() #46

aishniparab opened this issue Jun 26, 2017 · 16 comments

Comments

@aishniparab
Copy link

aishniparab commented Jun 26, 2017 •
edited

I ran the function to fetch the data and it has created a directory datasets/housing correctly, however I think it has trouble downloading housing.tgz file. Please help!

---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1317                 h.request(req.get_method(), req.selector, req.data, headers,
-> 1318                           encode_chunked=req.has_header('Transfer-encoding'))
   1319             except OSError as err: # timeout error

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in request(self, method, url, body, headers, encode_chunked)
   1238         """Send a complete request to the server."""
-> 1239         self._send_request(method, url, body, headers, encode_chunked)
   1240 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1284             body = _encode(body, 'body')
-> 1285         self.endheaders(body, encode_chunked=encode_chunked)
   1286 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in endheaders(self, message_body, encode_chunked)
   1233             raise CannotSendHeader()
-> 1234         self._send_output(message_body, encode_chunked=encode_chunked)
   1235 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in _send_output(self, message_body, encode_chunked)
   1025         del self._buffer[:]
-> 1026         self.send(msg)
   1027 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in send(self, data)
    963             if self.auto_open:
--> 964                 self.connect()
    965             else:

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py in connect(self)
   1399             self.sock = self._context.wrap_socket(self.sock,
-> 1400                                                   server_hostname=server_hostname)
   1401             if not self._context.check_hostname and self._check_hostname:

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    400                          server_hostname=server_hostname,
--> 401                          _context=self, _session=session)
    402 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py in __init__(self, sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, family, type, proto, fileno, suppress_ragged_eofs, npn_protocols, ciphers, server_hostname, _context, _session)
    807                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 808                     self.do_handshake()
    809 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py in do_handshake(self, block)
   1060                 self.settimeout(None)
-> 1061             self._sslobj.do_handshake()
   1062         finally:

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py in do_handshake(self)
    682         """Start the SSL/TLS handshake."""
--> 683         self._sslobj.do_handshake()
    684         if self.context.check_hostname:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
<ipython-input-28-09097de26f5a> in <module>()
----> 1 fetch_housing_data()

<ipython-input-27-fa2a4bf02df6> in fetch_housing_data(housing_url, housing_path)
     12         os.makedirs(housing_path)
     13     tgz_path = os.path.join(housing_path, "housing.tgz")
---> 14     urllib.request.urlretrieve(housing_url, tgz_path)
     15     housing_tgz = tarfile.open(tgz_path)
     16     housing_tgz.extractall(path=housing_path)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in urlretrieve(url, filename, reporthook, data)
    246     url_type, path = splittype(url)
    247 
--> 248     with contextlib.closing(urlopen(url, data)) as fp:
    249         headers = fp.info()
    250 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    221     else:
    222         opener = _opener
--> 223     return opener.open(url, data, timeout)
    224 
    225 def install_opener(opener):

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in open(self, fullurl, data, timeout)
    524             req = meth(req)
    525 
--> 526         response = self._open(req, data)
    527 
    528         # post-process response

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in _open(self, req, data)
    542         protocol = req.type
    543         result = self._call_chain(self.handle_open, protocol, protocol +
--> 544                                   '_open', req)
    545         if result:
    546             return result

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    502         for handler in handlers:
    503             func = getattr(handler, meth_name)
--> 504             result = func(*args)
    505             if result is not None:
    506                 return result

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in https_open(self, req)
   1359         def https_open(self, req):
   1360             return self.do_open(http.client.HTTPSConnection, req,
-> 1361                 context=self._context, check_hostname=self._check_hostname)
   1362 
   1363         https_request = AbstractHTTPHandler.do_request_

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1318                           encode_chunked=req.has_header('Transfer-encoding'))
   1319             except OSError as err: # timeout error
-> 1320                 raise URLError(err)
   1321             r = h.getresponse()
   1322         except:

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>
@ageron
Copy link
Owner

ageron commented Jun 27, 2017

Hi Aishni,

I believe this bug is due to the fact that Python 3.6 on MacOSX has no certificates at all (see the release notes), so it cannot verify the SSL certificate from GitHub's servers when trying to download housing.tgz. See the second answer to this StackOverflow question for details. The solution is to:

  • read /Applications/Python 3.6/ReadMe.rtf
  • the ReadMe will have you run /Applications/Python 3.6/Install Certificates.command which installs the certificates.

Alternatively, you can work around the issue by downloading the file yourself and placing it in the housing directory, then comment out the urlretrieve() line in the code, and run it again.

Hope this helps,
Aurélien

@ageron
Copy link
Owner

ageron commented Jun 29, 2017

Hi Aishni,

Did my answer solve your problem? May I close this issue?

Cheers,
Aurélien

@aishniparab
Copy link
Author

aishniparab commented Jun 29, 2017 via email

@ageron
Copy link
Owner

ageron commented Jun 29, 2017

Perfect, thanks for your feedback. :)

@ageron ageron closed this as completed Jun 29, 2017
@ageron ageron mentioned this issue Jul 18, 2017
Closed
@MarsForever MarsForever mentioned this issue Oct 1, 2017
Merged
This was referenced Apr 1, 2018
Closed
Open
@ClaudeCoulombe
Copy link

For Mac OS X

  1. Update to Python 3.6.5 using the native app installer downloaded from the official Python language website https://www.python.org/downloads/

I've found that the installer is taking care of updating the links and symlinks for the new Python a lot better than homebrew.

  1. Install a new certificate using "./Install Certificates.command" which is in the refreshed Python 3.6 directory

    cd "/Applications/Python 3.6/"
    sudo "./Install Certificates.command"

@purshipurshi2005
Copy link

Thanks the solution worked .

read /Applications/Python 3.6/ReadMe.rtf
the ReadMe will have you run /Applications/Python 3.6/Install Certificates.command which installs the certificates.

@Sunil1997
Copy link

Sunil1997 commented Jul 19, 2018 •
edited by ageron

Hey
i'm using fedora as os
it's give me error below
i'm using pycharm with envs of anaconda

WARNING:tensorflow:From /home/sunil/PycharmProjects/test/testFile.py:7: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
Extracting MNIST_data/train-images-idx3-ubyte.gz
WARNING:tensorflow:From /home/sunil/anaconda3/envs/condaEnvTest/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please write your own downloading logic.
WARNING:tensorflow:From /home/sunil/anaconda3/envs/condaEnvTest/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
WARNING:tensorflow:From /home/sunil/anaconda3/envs/condaEnvTest/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
WARNING:tensorflow:From /home/sunil/anaconda3/envs/condaEnvTest/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.one_hot on tensors.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
WARNING:tensorflow:From /home/sunil/anaconda3/envs/condaEnvTest/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.

@ageron
Copy link
Owner

ageron commented Jul 23, 2018

Hi @Sunil1997 ,
These are just warnings, not errors, you can ignore them for now. I will update the notebooks to use Keras' functionality for loading MNIST, because TensorFlow's functionality has been deprecated and is printing all these warnings. However, the code still works for now, so you can just ignore these warnings.

@alexhagen alexhagen mentioned this issue Oct 16, 2018
Closed
@jemberie
Copy link

jemberie commented Oct 1, 2019 •
edited by ageron

i am run the following on red hat Linux

In [12]: from keras.applications import VGG16                                   
In [13]: from keras import backend as K                                         
In [14]: model = VGG16(weights='imagenet', include_top=False)

and i found the following error

Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/anaconda3/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1317                 h.request(req.get_method(), req.selector, req.data, headers,
-> 1318                           encode_chunked=req.has_header('Transfer-encoding'))
   1319             except OSError as err: # timeout error

~/anaconda3/lib/python3.6/http/client.py in request(self, method, url, body, headers, encode_chunked)
   1253         """Send a complete request to the server."""
-> 1254         self._send_request(method, url, body, headers, encode_chunked)
   1255 

~/anaconda3/lib/python3.6/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1299             body = _encode(body, 'body')
-> 1300         self.endheaders(body, encode_chunked=encode_chunked)
   1301 

~/anaconda3/lib/python3.6/http/client.py in endheaders(self, message_body, encode_chunked)
   1248             raise CannotSendHeader()
-> 1249         self._send_output(message_body, encode_chunked=encode_chunked)
   1250 

~/anaconda3/lib/python3.6/http/client.py in _send_output(self, message_body, encode_chunked)
   1035         del self._buffer[:]
-> 1036         self.send(msg)
   1037 

~/anaconda3/lib/python3.6/http/client.py in send(self, data)
    973             if self.auto_open:
--> 974                 self.connect()
    975             else:

@ageron
Copy link
Owner

ageron commented Oct 5, 2019

Hi @jemberie ,

This code does not look like it comes from the book or from this repository. It looks like you're getting a timeout while downloading VGG16. Check your internet connection? Try from another computer? If nothing works, please ask on StackOverflow instead.

@alienintheheights
Copy link

This fixes it too -- added the ssl context workaround

import os
import tarfile
import urllib
import ssl

DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml2/master/"
HOUSING_PATH = os.path.join("datasets", "housing")
HOUSING_URL = DOWNLOAD_ROOT + "datasets/housing/housing.tgz"

def fetch_housing_data(housing_url=HOUSING_URL, housing_path=HOUSING_PATH):
    os.makedirs(housing_path, exist_ok=True)
    tgz_path = os.path.join(housing_path, "housing.tgz")
    context = ssl._create_unverified_context()
    response = urllib.request.urlopen(housing_url, context=context)
    with open(tgz_path, 'wb') as f:
        f.write(response.read())
    housing_tgz = tarfile.open(tgz_path)
    housing_tgz.extractall(path=housing_path)
    housing_tgz.close()

@clenchedteeth clenchedteeth mentioned this issue Feb 18, 2021
Closed
@adarshraghav
Copy link

--This answer is for MACOS--
Hi,
For python 3.9 and above, just go to Applications -> Python x.x -> there will be a .command file to install certificates, just click on that and you're done.

@OpenWiseFound
Copy link

OpenWiseFound commented May 15, 2023 via email

@Mauricio9
Copy link

Mauricio9 commented May 15, 2023 via email

@OpenWiseFound
Copy link

OpenWiseFound commented May 15, 2023 via email

@Mauricio9
Copy link

Mauricio9 commented May 15, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@ageron @alienintheheights @aishniparab @purshipurshi2005 @ClaudeCoulombe @jemberie @Mauricio9 @OpenWiseFound @Sunil1997 @adarshraghav