SSL certificate verification.

AUTHORS.txt

@@ -25,6 +25,7 @@ Ian Bicking
 Igor Sobreira
 Ionel Maries Cristian
 Jakub Vysoky
+James Cleveland
 Jannis Leidel
 Jay Graves
 John-Scott Atlakson

CHANGES.txt

@@ -4,6 +4,9 @@ Changelog
 develop (unreleased)
 --------------------
 
+* SSL Cert Verification; Make https the default for PyPI access.
+  Thanks James Cleveland, Giovanni Bajo and many others (Pull #789).
+
 * Added "pip list" for listing installed packages and the latest version
   available. Thanks Rafael Caricio, Miguel Araujo, Dmitry Gladkov (Pull #752)
 

LICENSE.txt

@@ -18,3 +18,22 @@ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
 LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
 OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+
+License for Bundle of CA Root Certificates (pip/cacert.pem)
+===========================================================
+
+This library is free software; you can redistribute it and/or
+modify it under the terms of the GNU Lesser General Public
+License as published by the Free Software Foundation; either
+version 2.1 of the License, or (at your option) any later version.
+
+This library is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+Lesser General Public License for more details.
+
+You should have received a copy of the GNU Lesser General Public
+License along with this library; if not, write to the Free Software
+Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+02110-1301

MANIFEST.in

@@ -2,6 +2,7 @@ include AUTHORS.txt
 include LICENSE.txt
 include CHANGES.txt
 include PROJECT.txt
+include pip/cacert.pem
 recursive-include docs *.txt
 recursive-include docs *.html
 recursive-exclude docs/_build *.txt

docs/index.txt

@@ -6,7 +6,7 @@ A tool for installing and managing Python packages.
 `Mailing list <http://groups.google.com/group/python-virtualenv>`_ ``|``
 `Issues <https://github.com/pypa/pip/issues>`_ ``|``
 `Github <https://github.com/pypa/pip>`_ ``|``
-`PyPI <http://pypi.python.org/pypi/pip/>`_ ``|``
+`PyPI <https://pypi.python.org/pypi/pip/>`_ ``|``
 irc:#pip
 
 

docs/installing.txt

@@ -3,11 +3,21 @@
 Installation
 ============
 
-Python Support
---------------
+.. warning::
+
+    Prior to version 1.3, pip did not use SSL for downloading packages from PyPI, and thus left
+    users more vulnerable to security threats. We advise installing at least version 1.3.
+    If you're using `virtualenv <http://www.virtualenv.org>`_ to install pip, we advise installing
+    at least version 1.9, which contains pip version 1.3.
+
+
+Python & OS Support
+-------------------
 
 pip works with CPython versions 2.5, 2.6, 2.7, 3.1, 3.2, 3.3 and also pypy.
 
+pip works on Unix/Linux, OS X, and Windows.
+
 
 Using virtualenv
 ----------------
@@ -33,40 +43,50 @@ Installing Globally
 pip can be installed globally in order to manage global packages.
 Often this requires the installation to be performed as root.
 
-Prerequisites
-+++++++++++++
+.. warning::
+
+    We advise against using easy_install to install pip, because easy_install
+    does not download from PyPI over SSL, so the installation might be insecure.
+    Since pip can then be used to install packages (which execute code on
+    your computer), it is better to go through a trusted path.
+
 
-A global install of pip requires either `setuptools <http://pypi.python.org/pypi/setuptools>`_
-or `distribute <http://pypi.python.org/pypi/distribute>`_ to be installed globally as well.
+Requirements
+++++++++++++
 
-In many cases, these can be installed using your OS package manager.
+pip requires either `setuptools <https://pypi.python.org/pypi/setuptools>`_
+or `distribute <https://pypi.python.org/pypi/distribute>`_.
 
-See the `Distribute Install Instructions <http://pypi.python.org/pypi/distribute/>`_ or the
-`Setuptools Install Instructions <http://pypi.python.org/pypi/setuptools#installation-instructions>`_
+See the `Distribute Install Instructions <https://pypi.python.org/pypi/distribute/>`_ or the
+`Setuptools Install Instructions <https://pypi.python.org/pypi/setuptools#installation-instructions>`_
+
+If installing pip using a linux package manager, these requirements will be installed for you.
 
 .. warning::
 
     If you are using Python 3.X you **must** use distribute; setuptools doesn't
     support Python 3.X.
 
+
 Using get-pip
 +++++++++++++
 
-Download `get-pip.py <https://raw.github.com/pypa/pip/master/contrib/get-pip.py>`_
-and execute it using Python.  This will only install pip, not the prerequisites.
+After installing the requirements:
 
 ::
 
  $ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
  $ [sudo] python get-pip.py
 
 
-From source
-+++++++++++
+Installing from source
+++++++++++++++++++++++
+
+After installing the requirements:
 
 ::
 
- $ curl -O http://pypi.python.org/packages/source/p/pip/pip-X.X.tar.gz
+ $ curl -O https://pypi.python.org/packages/source/p/pip/pip-X.X.tar.gz
  $ tar xvfz pip-X.X.tar.gz
  $ cd pip-X.X
  $ [sudo] python setup.py install

docs/logic.txt

@@ -151,17 +151,73 @@ pip offers a set of :ref:`Package Index Options <Package Index Options>` for mod
 See the :ref:`pip install Examples<pip install Examples>`.
 
 
+.. _`SSL Certificate Verification`:
+
+SSL Certificate Verification
+============================
+
+Starting with v1.3, pip provides SSL certificate verification over https, for the purpose
+of providing secure, certified downloads from PyPI.
+
+This is supported by default in all Python versions pip supports, except Python 2.5.
+
+Python 2.5 users can :ref:`install an SSL backport <SSL Backport>`, which provides ssl support for older pythons.
+Pip does not try to install this automatically because it requires a compiler, which not all systems will have.
+
+Although not recommended, Python 2.5 users who are unable to install ssl, can use the global option,
+``--insecure``, to allow access to PyPI w/o attempting SSL certificate verification. This option will only be visible
+when ssl is not importable.  This is *not* a general option.
+
+
+.. _`SSL Backport`:
+
+Installing the SSL Backport
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. warning::
+
+    We advise against using ``pip`` itself to install the ssl backport, because it won't be secure
+    until *after* installing ssl.  Likewise, ``easy_install`` is not advised, because it
+    does not currently support ssl.
+
+
+1. Download the ssl archive:
+
+  * Using a Browser:
+
+    1. Go to `this url <https://pypi.python.org/pypi/ssl/1.15>`_.
+    2. Confirm the identity of the site is valid.
+       Most browsers provide this information to the left of the URL bar in the form of padlock icon that you can click on to confirm the site is verified.
+    3. Scroll down, and click to download ``ssl-1.15.tar.gz``.
+
+  * Using curl, which supports ssl certificate verification:
+     ::
+
+      $ curl -O https://pypi.python.org/packages/source/s/ssl/ssl-1.15.tar.gz
+
+2. Confirm the md5sum:
+   ::
+     $ md5sum ssl-1.15.tar.gz
+     81ea8a1175e437b4c769ae65b3290e0c  ssl-1.15.tar.gz
+
+3. Unpack the archive, and change into the ``ssl-1.15`` directory.
+4. Run: ``python setup.py install``.
+
+
 Hash Verification
 =================
 
-PyPI provides an md5 hash of a package by having the link to the
-package include an #md5=<hash>.
+PyPI provides md5 hashes in the hash fragment of package download urls.
 
 pip supports checking this, as well as any of the
 guaranteed hashlib algorithms (sha1, sha224, sha384, sha256, sha512, md5).
 
 The hash fragment is case sensitive (i.e. sha1 not SHA1).
 
+This check is only intended to provide basic download corruption protection.
+It is not intended to provide security against tampering. For that,
+see :ref:`SSL Certificate Verification`
+
 
 Download Cache
 ==============

pip/backwardcompat.py → pip/backwardcompat/__init__.py

@@ -112,3 +112,24 @@ def home_lib(home):
     else:
         lib = os.path.join('lib', 'python')
     return os.path.join(home, lib)
+
+
+## py25 has no builtin ssl module
+## only >=py32 has ssl.match_hostname and ssl.CertificateError
+try:
+    import ssl
+    try:
+        from ssl import match_hostname, CertificateError
+    except ImportError:
+        from pip.backwardcompat.ssl_match_hostname import match_hostname, CertificateError
+except ImportError:
+    ssl = None
+
+
+# patch for py25 socket to work with http://pypi.python.org/pypi/ssl/
+import socket
+if not hasattr(socket, 'create_connection'): # for Python 2.5
+    # monkey-patch socket module
+    from pip.backwardcompat.socket_create_connection import create_connection
+    socket.create_connection = create_connection
+

pip/backwardcompat/socket_create_connection.py

@@ -0,0 +1,44 @@
+"""
+patch for py25 socket to work with http://pypi.python.org/pypi/ssl/
+copy-paste from py2.6 stdlib socket.py
+https://gist.github.com/zed/1347055
+"""
+import socket
+import sys
+
+_GLOBAL_DEFAULT_TIMEOUT = getattr(socket, '_GLOBAL_DEFAULT_TIMEOUT', object())
+def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
+                      source_address=None):
+    """Connect to *address* and return the socket object.
+
+    Convenience function.  Connect to *address* (a 2-tuple ``(host,
+    port)``) and return the socket object.  Passing the optional
+    *timeout* parameter will set the timeout on the socket instance
+    before attempting to connect.  If no *timeout* is supplied, the
+    global default timeout setting returned by :func:`getdefaulttimeout`
+    is used.
+    """
+
+    host, port = address
+    err = None
+    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
+        af, socktype, proto, canonname, sa = res
+        sock = None
+        try:
+            sock = socket.socket(af, socktype, proto)
+            if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
+                sock.settimeout(timeout)
+            if source_address:
+                sock.bind(source_address)
+            sock.connect(sa)
+            return sock
+
+        except socket.error:
+            err = sys.exc_info()[1]
+            if sock is not None:
+                sock.close()
+
+    if err is not None:
+        raise err
+    else:
+        raise socket.error("getaddrinfo returns an empty list")

pip/backwardcompat/ssl_match_hostname.py

@@ -0,0 +1,60 @@
+"""The match_hostname() function from Python 3.2, essential when using SSL."""
+
+import re
+
+__version__ = '3.2a3'
+
+class CertificateError(ValueError):
+    pass
+
+def _dnsname_to_pat(dn):
+    pats = []
+    for frag in dn.split(r'.'):
+        if frag == '*':
+            # When '*' is a fragment by itself, it matches a non-empty dotless
+            # fragment.
+            pats.append('[^.]+')
+        else:
+            # Otherwise, '*' matches any dotless fragment.
+            frag = re.escape(frag)
+            pats.append(frag.replace(r'\*', '[^.]*'))
+    return re.compile(r'\A' + r'\.'.join(pats) + r'\Z', re.IGNORECASE)
+
+def match_hostname(cert, hostname):
+    """Verify that *cert* (in decoded format as returned by
+    SSLSocket.getpeercert()) matches the *hostname*.  RFC 2818 rules
+    are mostly followed, but IP addresses are not accepted for *hostname*.
+
+    CertificateError is raised on failure. On success, the function
+    returns nothing.
+    """
+    if not cert:
+        raise ValueError("empty or no certificate")
+    dnsnames = []
+    san = cert.get('subjectAltName', ())
+    for key, value in san:
+        if key == 'DNS':
+            if _dnsname_to_pat(value).match(hostname):
+                return
+            dnsnames.append(value)
+    if not san:
+        # The subject is only checked when subjectAltName is empty
+        for sub in cert.get('subject', ()):
+            for key, value in sub:
+                # XXX according to RFC 2818, the most specific Common Name
+                # must be used.
+                if key == 'commonName':
+                    if _dnsname_to_pat(value).match(hostname):
+                        return
+                    dnsnames.append(value)
+    if len(dnsnames) > 1:
+        raise CertificateError("hostname %r "
+            "doesn't match either of %s"
+            % (hostname, ', '.join(map(repr, dnsnames))))
+    elif len(dnsnames) == 1:
+        raise CertificateError("hostname %r "
+            "doesn't match %r"
+            % (hostname, dnsnames[0]))
+    else:
+        raise CertificateError("no appropriate commonName or "
+            "subjectAltName fields were found")