Skip to content
This repository has been archived by the owner on Jul 16, 2020. It is now read-only.

PyPI compromise and pip with TUF

Jump to bottom
Trishank Karthik Kuppusamy edited this page Sep 21, 2013 · 10 revisions

In the event of a PyPI compromise, we want to see how pip is affected. When and how would an attacker be able to coax pip into install malicious packages?

First, we set up the virtual environment (for cleanroom testing) and install pip-with-TUF:

$ cd  /tmp
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
$ tar xvfz virtualenv-1.10.1.tar.gz
$ python virtualenv-1.10.1/virtualenv.py --no-site-packages pypi-compromise-with-tuf
$ source pypi-compromise-with-tuf/bin/activate
$ sudo apt-get install python-dev libgmp-dev
$ pip install --upgrade https://github.com/theupdateframework/tuf/archive/v0.7.5.zip
$ pip install --upgrade https://github.com/theupdateframework/pip/archive/trishank.poly.edu.zip

First, suppose that we introduce the FooBar package on PyPI-with-TUF by having the "unclaimed" targets sign for it. This means that the developers of FooBar have not yet "claimed" ownership (and thus responsibility) for the package, so PyPI will use an online key to sign for all "unclaimed" targets. (This key is online to allow for continuous release of "unclaimed" PyPI targets.) This is the lowest level of security for a package on PyPI-with-TUF: we will protect the package from being tampered by a mirror or a CDN, but we cannot protect it from being tampered by an attacker who has compromised PyPI.

# POINT TO REPOSITORY WITH INTACT TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.unclaimed.good/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install FooBar
Downloading/unpacking FooBar
  Downloading FooBar-0.1.tar.gz (unknown size): 539bytes downloaded
  Running setup.py egg_info for package FooBar
    
Installing collected packages: FooBar
  Running setup.py install for FooBar
    FooBar 0.1
    
Successfully installed FooBar
Cleaning up...

Let us see what happens when an attacker compromises PyPI and tampers with the FooBar package targets and signs for them with the online "unclaimed" targets role key:

# POINT TO REPOSITORY WITH TAMPERED TARGETS AND TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.unclaimed.bad/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install --upgrade FooBar
Downloading/unpacking FooBar from https://pypi.python.org/packages/source/F/FooBar/FooBar-0.2.tar.gz#md5=10831baad99c6acbcd59103b1099d13c
  Downloading FooBar-0.2.tar.gz (unknown size): 552bytes downloaded
  Running setup.py egg_info for package FooBar
    
Installing collected packages: FooBar
  Found existing installation: FooBar 0.1
    Uninstalling FooBar:
      Successfully uninstalled FooBar
  Running setup.py install for FooBar
    TAMPERED FooBar 0.2
    
Successfully installed FooBar
Cleaning up...

In summary, the "unclaimed" targets role will allow for continuous release of PyPI packages, as is the case of PyPI-without-TUF today. However, unlike PyPI-without-TUF, it will protect all "unclaimed" PyPI packages from attacks by mirrors or CDNs. (An "unclaimed" PyPI package is one for which its developers have not registered their own signing keys to sign for all TUF metadata about that package.) Nevertheless, since the "unclaimed" targets role key is online to allow continuous release, any compromise of PyPI will not protect "unclaimed" targets. (We will soon see how to protect PyPI packages even against a compromise of PyPI with the "claimed" targets role.)

Now, suppose that developers of the FooBar package register their signing keys (to sign for all FooBar targets) with PyPI-with-TUF. PyPI-with-TUF will then delegate all FooBar targets from the "recently-claimed" targets role to the FooBar targets role. If a target is available in both the "recently-claimed" and "unclaimed" targets roles, "recently-claimed" would be preferred over "unclaimed" to be responsible for the target:

# POINT TO REPOSITORY WITH INTACT TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.recently-claimed.good/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install --upgrade FooBar
Downloading/unpacking FooBar from https://pypi.python.org/packages/source/F/FooBar/FooBar-0.3.tar.gz#md5=eb534887d5d531c594715bd929c6c789
  Downloading FooBar-0.3.tar.gz (unknown size): 537bytes downloaded
  Running setup.py egg_info for package FooBar
    
Installing collected packages: FooBar
  Found existing installation: FooBar 0.2
    Uninstalling FooBar:
      Successfully uninstalled FooBar
  Running setup.py install for FooBar
    FooBar 0.3
    
Successfully installed FooBar
Cleaning up...

However, it is important to note that "recently-claimed" does not offer less or more security than "unclaimed". "recently-claimed" allows for the continuous release of PyPI packages for which their respective developers have registered their own package-signing keys; therefore, the signing key for "recently-claimed", too, must remain online. Here is an example of an attacker who has compromised PyPI and modifies the delegation of the FooBar package in "recently-claimed":

# POINT TO REPOSITORY WITH TAMPERED TARGETS AND TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.recently-claimed.bad/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install --upgrade FooBar
Downloading/unpacking FooBar from https://pypi.python.org/packages/source/F/FooBar/FooBar-0.4.tar.gz#md5=5d21b9feebf2118d3e0916cbfce22176
  Downloading FooBar-0.4.tar.gz (unknown size): 548bytes downloaded
  Running setup.py egg_info for package FooBar
    
Installing collected packages: FooBar
  Found existing installation: FooBar 0.3
    Uninstalling FooBar:
      Successfully uninstalled FooBar
  Running setup.py install for FooBar
    TAMPERED FooBar 0.4
    
Successfully installed FooBar
Cleaning up...

What is the point of "recently-claimed", then? It exists so that PyPI-with-TUF may continuously release packages for which developers have registered their own package-signing keys. Once in a while, PyPI-with-TUF will strongly secure the delegations in "recently-claimed" by promoting them to the "claimed" targets role.

Suppose that PyPI-with-TUF has promoted the FooBar package from "recently-claimed" to "claimed". If a target is available in the "claimed", "recently-claimed", and "unclaimed" targets roles, then "claimed" would be the most preferred role to be responsible for the target:

# POINT TO REPOSITORY WITH INTACT TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.claimed.good/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install --upgrade FooBar
Downloading/unpacking FooBar from https://pypi.python.org/packages/source/F/FooBar/FooBar-0.5.tar.gz#md5=02ccbb78795c3671b7f3b856ec9a6951
  Downloading FooBar-0.5.tar.gz (unknown size): 539bytes downloaded
  Running setup.py egg_info for package FooBar
    
Installing collected packages: FooBar
  Found existing installation: FooBar 0.4
    Uninstalling FooBar:
      Successfully uninstalled FooBar
  Running setup.py install for FooBar
    FooBar 0.5
    
Successfully installed FooBar
Cleaning up...

In this case, the pip-with-TUF user trusted PyPI to provide secure metadata about which package-signing keys the developers of the FooBar package intended for the package. This delegation metadata is secure because the "claimed" targets role key is offline.

Up to now, the story has looked similar for the case where the user uses pip without TUF. Suppose that an attacker compromised PyPI and tampered with the TUF metadata for the FooBar package:

# POINT TO REPOSITORY WITH TAMPERED TARGETS AND TUF METADATA.
$ curl http://mirror1.poly.edu/test-pip/pypi-compromise/repository.claimed.bad/tuf.interposition.json -o pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/tuf.interposition.json

$ pip install --upgrade FooBar
Downloading/unpacking FooBar
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/tmp/pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/tmp/pypi-compromise-with-tuf/local/lib/python2.7/site-packages/pip/commands/install.py", line 270, in run
    sys.exit('TUF stopped the update due to an error:\n'+str(error))
SystemExit: TUF stopped the update due to an error:
No working mirror was found:
  mirror1.poly.edu: targets/claimed/FooBar metadata has bad signature!

Storing complete log in /home/trishank/.pip/pip.log

Finally, we see that the FooBar package is secure from attacks on the PyPI-with-TUF infrastructure because the "root", "targets", "targets/claimed", "targets/claimed/FooBar" keys are all offline. Compare this with the situation where pip or PyPI works without TUF.

Clone this wiki locally