When developing multiple Python projects concurrently, each having its own separate set of modules/packages which might conflict, and sometimes even its own version of Python, it is very hard to keep track of which Python interpreter and which set of libraries go with each project.
As for Python selection, an excellent tool called Pythonz exists to simplify the installation, usage, and reinstallation of Python versions.
As for the package sets, another industry-standard tool called virtualenv enables you to use "virtual pythons" on top of your existing installation to isolate environments. To simplify the workflow, a wrapper exists that enables you to quickly switch between virtual environments.
The typical scenario (at least for some developers) is working on several projects concurrently - some may be for work, some personal, etc. Each set of projects needs its own virtualenv. However, the way virtualenv works, you must remember to activate the "right" environment before working on a project.
This process may be familiar to you as this sequence of actions:
- start a shell
workonyour favorite virtualenv
python setup.py developyour project and start working on it.
Although virtualenvwrapper and virtualenv work quite well, the above sequence has several annoying problems when switching projects and/or shell sessions.
Problem no. 1: Wrong Environments
Running the wrong python can be a painful mistake. You can install your project in the global python installation (which means a lot of dependencies creeping in). Also one project being developed in one environment (via
setup.py develop) can be a dependency in another environment, which can get you wasting a whole lot of time trying to regain control over your working environment.
A quick look at the above set of actions reveals a great potential for getting the wrong environment by mistake. It gets even worse when using "console scripts" -- for instance, if you have
nose installed in your global Python installation, then the
nosetests script will be in your $PATH somewhere. However, if you try to use it to test a project being developed in a virtual environment, it will fail importing stuff.
Problem no. 2: IDEs
Even if you got to activating your "right" environment, that only solves your problems for the current shell session. Once you start a new shell you're forced to deal with the same issue again.
It gets even worse when you try to integrate various scripts and tools into your editors/IDEs. If you want to run a script from your project with
emacs, for instance, you can't just run it as-is, because it relies on the virtual python installation that it was installed on. This gets you to mess around with your configuration schemes on a per-project basis hoping to get it right, and is generally a headache.
Problem no. 3: Responsiveness
Many solutions involve a shell mode of some sort to switch your default python interpreter (like
pb switch and
virtualenvwrapper.sh). They are great, but sometimes they take some time to execute. This causes your
.zshrc to take an extra second or so, and sometimes even more. I don't know about you, but my workflow includes opening and closing many tabs and panes, so this can get pretty annoying pretty fast.
pytoggle is a very simple utility meant to be used alongside other tools like
pythonz. It borrows the idea of direnv and adapts it to the problems discussed above. It is a "smart shortcut" that deduces the version of python you'd like to run and runs it for you.
Step 1: Clone the Project
That's an easy one (NB the path used for the repo):
$ git clone https://github.com/vmalloc/pytoggle.git ~/.pytoggle
Make sure that the shebang (
#!) on top of
~/.pytoggle/pytoggle makes sense in your environment.
pytoggle itself is python-based, so it needs a concrete Python interpreter to run. Once you get that right, just type:
$ ~/.pytoggle/pytoggle install
Step 2: Fix your
Include this in your shell profile file (
.bashrc for bash,
.zshrc for zsh, etc.):
Step 3: Specifying your configuration
pytoggle works best served with
pythonz. Let's say your project resides in
~/projects/proj1. You can install a version of python to act as your default, and create a virtualenv for your project:
$ ~/.pythonz/pythons/CPython-2.7.5/bin/virtualenv proj1_venv
The above will normally create a virtualenv under
proj1_venv/. Now edit a file named
.pytoggle.cfg and put it anywhere above your project directory (for instance, in
[bin_dir] path = /path/to/proj1_venv/bin
That's it! Now when you run python from within the
proj1 dir, you'll get the "right" python for you:
$ cd ~/projects/proj1 $ python Python 2.7.3 (default, Jul 23 2012, 21:34:02) Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.executable '/path/to/proj1_venv/bin/python'
Also, when you run a python script in your project directory, the right python interpreter will be used, even if you're not in that directory:
$ cd /tmp $ python ~/projects/proj1/script.py # this will run the proj1 venv python
Step 4 (optional): Create More Aliases
By default, the
pip aliases are created for you. If you would like to alias more commands to be toggled between environments, just run
$ pytoggle alias nosetests
The above will make
nosetests choose the "right" virtual environment as well.