Skip to content

Python Virtual Environments

Harry Hummel edited this page Sep 21, 2019 · 14 revisions

War of the worlds

We are using pipenv to manage virtual environments within the Python world. The details are here https://github.com/pyenv/pyenv#homebrew-on-macos. There are several interacting environments on your computer, which can be confusing:

  • The operating system. You install into this environment using brew install your-package or apt install your-package. This is the process you use to install pipenv on your system, as well as the Python language itself. More about that below.
  • Docker and docker-compose. We are isolating the postgres database inside its own environment using docker. Were you to have installed postgres independently on your system, e.g. using brew install postgresql, then there would be two different versions of postgres and the app might connect the the wrong one. This has happened to the author.
  • The python environment, including Django and Django Rest Framework. This is where the code written in Python goes and where pipenv comes into the picture.
  • The frontend environment, written in Javascript, is controlled by Yarn.

What pipenv does

Pipenv creates an isolated Python environment on your computer, automatically downloading the required Python packages. There are lots of competing packages that do parts of this: pip, easy install, virtual environment, virtual environment wrapper, Anaconda. We can hope pipenv is the one package manager and virtual environment to rule them all!

  • It keeps track of your dependencies in the Pipfile and Pipfile.lock.
  • When you are in the environment, it sees the packages for that environment. So if you've set up the environment for Python 3.7 but your system version is Python 2.7, inside the pipenv if you enter python, it loads Python 3.7

How do you install pipenv?

You have to install pipenv, the version of Python, and perhaps pyenv:

  • Install pipenv, e.g brew install pipenv
  • Install the python language, e.g. brew install python
  • You might want to install pyenv, e.g. brew install pyenv. This lets pipenv install the correct version of Python for you. If you have been specifying python3.7 in the Pipfile, and you then want to update to version 3.8, pyenv does this for you. It works with pipenv, rather than conflicting with it. From the pipenv documentation:

Automatically install required Pythons, if pyenv is available.

How do you use pipenv?

Go to the project root file, then:

  • Enter pipenv shell. This opens an existing virtual environment using the Pipfile if it exists, or creates a new one. Do your python development inside this shell.
  • If you need to install a python package, do it inside the shell using pipenv install your_package. The Pipfile is automatically updated.
  • Enter exit to leave the shell.
  • Enter pipenv sync to update the virtual environment.

What if something goes wrong?

Ah. This has happened to the author, too. Try this:

  • Make sure you haven't created a pipenv inside a pipenv. There should be a Pipfile at the root, but not inside. If there is one, remove it.
  • Exit the shell at the project root and try pipenv lock --clear
  • Try pipenv --rm to rebuild the virtual environment.