Matplotlib fails inside virtualenv

[jorgehatccrma]

Matplotlib doesn't show the figure window from within a virtualenv. I've tried many combinations (e.g. intalling using pip, using easy_install, compiling from source, using --system-site-packages, different versions, etc.) with no luck. The test that makes me think is a virtualenv issue is that if I try to use my working system-wide matplotlib from virtualenv, I don't see a figure. In other words, if I run this:

[~]$ python plotTest.py

it works ok (shows the figure). Then, if I create a virtualenv with --system-site-packages and run the same script, it doesn't work. That is:

[~]$ virtualenv --system-site-packages vEnv
[~]$ source vEnv/bin/activate
(vEnv)[~]$ python plotTest.py

fails (it hangs and doesn't even return). I need to kill the process.
Btw, the plotTest.py if the following:

import matplotlib
import matplotlib.pyplot as plt

print "Backend: ", matplotlib.rcParams['backend']

plt.plot([1,2,1])
plt.show()

FWIW, in both cases I do see the line printing MacOSX as the backend.

[anddam]

Most likely you have a bad-behaving (it won't pop up, it won't take focus) window behind your current terminal. Try hiding all apps, then run your test and then move or resize the terminal window (mission control and other exposé-like stuff won't help).

See matplotlib/matplotlib#2881 and related.

This is most likely due to virtualenv not reproducing the Framework-install, I'm trying to fix that myself.

[nils-werner]

Any updates on this?

Has anybody located the issue (yup, no Framework installs, but where exactly does it fail?) already?

[nils-werner]

Just for sake of documentation, http://wiki.wxpython.org/wxPythonVirtualenvOnMac solves this issue by calling systemwide python with a PYTHONHOME environment variable.

[nils-werner]

Note that copying your global python executable into the virtualenv bin/ directory as suggested by some does not help. It breaks the virtualenv by calling python as if there as no virtualenv to begin with.

[anddam]

No update on my side, actually back then I needed matplotlib for a brief while and in order to produce some plot so I didn't spent time on that.

You may want to provide a reproducible example to help developers tracking the bug, see http://sscce.org

[nils-werner]

Reproducible example:

The following commandline script exhibits the bug:

virtualenv tmp
source tmp/bin/activate
pip install matplotlib
python -c 'import matplotlib.pyplot as plt; plt.plot(); plt.show()'

• The Matplotlib window will hide behind Terminal
• Cmd+Tab will not select the window
• All Shortcuts typed while clicking around in the window go to Terminal instead

The following commandline script does not exhibit the bug:

pip install matplotlib
python -c 'import matplotlib.pyplot as plt; plt.plot(); plt.show()'

• The Matplotlib window will not hide behind Terminal
• Cmd+Tab will select the window
• All Shortcuts typed while clicking around in the window are recognised

[nils-werner]

I can confirm that the code from gldnspud/virtualenv-pythonw-osx does fix the issue I am having. Is there a way we can integrate it into mainline virtualenv?

[agiledata]

We've had the same problem with Pygame Zero, running in a virtualenv on a Mac. We have a workaround, described in this thread. It means substituting the system Framework Python for the virtualenv one that can't handle windowing properly on a Mac.

[anddam]

@agiledata how's that going to work? On my Yosemite system I don't have '/Library/Frameworks/Python.framework/Versions/' at all, and anyway the system-shipped Python framework doesn't have 3.x

c.f. https://bitbucket.org/lordmauve/pgzero/pull-request/20/fix-for-windowing-problem-for-pygame-zero/diff#Lpgzero/runner.pyT44

[agiledata]

@anddam We made the fix for Pygame Zero, where having Python 3 installed is a requirement. But the principle is the same, whichever Python version you want to substitute, just run a full framework Python with the environment from virtualenv.

[nils-werner]

As of matplotlib 1.5.0 you are required to use ugly hacks to make it work in virtualenvs. Can we please fix this issue already?

[jenshnielsen]

@nils-werner You have always needed ugly workarounds to make it work. It's just now that we added a check and fail hard rather than trying to open a broken window that is borderline useless and can't be put in the foreground.

[nils-werner]

@jenshnielsen I am not blaming matplotlib but virtualenv.

[optimuspaul]

I'm more inclined at this point to blame matplotlib to be honest. I'm not even using virtualenv right now and I'm seeing all the same issues after trying all their hacks to make it work with virtualenv.

[jenshnielsen]

@optimuspaul I am sorry but you are simply wrong. As it is there is no way you can run a Framework build inside a virtualenv. It simply doesn't work. That means that you CANNOT interact reliably with the GUI frameworks inside a virtual env without work arounds.

The work around we have given are only meant to be used inside a virtual env. How you managed to apply them outside is beyond me but it must be a user error at your side. The work arounds do work inside virtual envs and that has been confirmed by several users.

If you want anyone to to help you with your issue you better provide more than a rant without details claiming that we broke stuff. We actually did spent time investigating this and trying to fix the issue for users.

[jenshnielsen]

And if you are seeing the issue outside a virtual env without workarounds then it probably means that your python build is just not a framework build in which case you will need to reinstall python as a framework build if you want to interact with OSX gui frameworks.

[wlsherica]

Finally I solved this problem by this article, http://www.wirywolf.com/2016/01/pyplot-in-jupyter-inside-pyenv-on-el-capitan.html

It works!!

[grantstephens]

@wisherica It is still a work around and does not work out the box.
+1 on getting this fixed

[fromneo]

I'm having issues with this on Python 3.5. Installed the frameworkpython script with pyver=3.5 in bin but I still get the backend error issue. Any clues? It works fine when I do this with Python 2.7 but not on Python 3.5

[jenshnielsen]

Im using the following slightly tweeked from the original which works with both python 2.7 and 3.5. Both of the default non virtual env are in /usr/local/bin

function frameworkpython {
    if [[ ! -z "$VIRTUAL_ENV" ]]; then
        #test if we are running python 2 or 3
        pyversion=`python -V 2>&1 | cut -d ' '  -f2 | cut -d. -f1,2`
        PYTHONHOME=$VIRTUAL_ENV /usr/local/bin/python$pyversion "$@"
    else
        echo "Not running in a virtualenv use regular python"
    fi
}

[fromneo]

I removed the frameworkpython script from my virtualenv /bin/. And put your function in my .zshrc file. Then I sourced it and tried running frameworkpython ~/.virtualenvs/py3/bin/jupyter-notebook. Now I am able to launch the notebook, but I am still unable to run import matplotlib.pyplot. The framework error still appears. There is a Python 3.5 in my /usr/local/bin folder.

[jenshnielsen]

Can you run matplotlib outside the virtualenv. It is possible that /usr/local/bin/python3.5 is not in it self a frameworkbuild of python. How did you install python3?

[fromneo]

I ran the python3.5 interpreter, imported matplotlib.pyplot and plotted a random graph. It shows up when done outside the virtual environment. I downloaded python 3.5 from this link https://www.python.org/downloads/release/python-351/ and installed it.

[jenshnielsen]

I just installed python3.5.1 from python.org and it works for me. Both versions of the script basically amount to:

export PYTHONHOME=$VIRTUAL_ENV
/pathtononvirtualenvpython test.py

were test.py is simply

import matplotlib.pyplot as plt
plt.plot(range(10))
plt.show()

Sorry to not be more helpful

[fromneo]

Thanks for your help so far! I can pinpoint exactly where its not working. Frameworkpython works with ipython in the terminal and with the method you mentioned. However, when I run frameworkpython jupyter-notebook and then run import matplotlib.pyplot in the ipython notebook itself, that's where I get the backend error.

[fromneo]

Finally got it to work using @wlsherica method. I don't know enough about Python to say but it could be specific to running matplotlib.pyplot in jupyter-notebook on OSX El Capitan. Thanks for your help @wisherica and @jenshnielsen!

[underyx]

I'm hitting the same issue, just in a way more annoying way. I'm trying to run the unit tests testing part of a Python module with tox. Tox automatically creates a virtualenv for this. Now:

1. The tox config cannot make assumptions about other developers' environments, so I can't really do anything the above solutions describe.
2. I'm just trying to run unit tests dammit, I don't need matplotlib to draw anything on the screen. The only thing I'm doing with matplotlib here is importing it. Why is it failing at import time instead of when being required to use one of these problematic frameworks?

[jenshnielsen]

Because matplotlib sets the backend at import time of pyplot. If you want a different backend I suggest selecting it before importing it or configuring another default backend.

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

[grantstephens]

You can also set the default backend in the matplotlibrc file if you can't edit your unit test files.

[underyx]

Yep, I've just found this solution actually just now @jenshnielsen @RexFuzzle, thanks for the help! Regardless, while I understand that I basically know nothing about the inner workings of matplotlib, I'm not happy about the fact that it purposefully raises an exception when imported.

[mattayes]

Is this solved? If so, can someone close it?

[jenshnielsen]

It is absolutely Not Solved. Matplotlib has documented various workarounds. The best one is to not use virtualenv but pyvenv which works correctly

[MircoT]

There is someone that use also virtualenvwrapper?

In that case I used a suggested workaround that matplotlib has in its FAQ that is create the environment with pyvenv (of course if you need Python 3). The interesting thing is that if you create the environment with pyvenv inside the WORKON_HOME folder virtualenvwrapper still works fine and I can use matplotlib without problem inside that environment.

Hope this thing can help and that I'm not off topic.

[aleksijohansson]

Pyenv (not pyvenv) and virtualenv are two different levels of sandboxing. I'm running both, pyenv to define the python version per project and pipenv (which uses virtualenv) to define modules per project. Pyenv doesn't provide a way to have multiple instances of the same version of python. It would be really good to get virtualenv working with properly. Any updates on this?

[jenshnielsen]

It works fine with pyenv virtualenv combined with pyenv provided that you remember to build the pyenv python version as a framework build as documented here It also works with venv instead of virtualenv for python3 which I think is what was meant by "pyvenv" above.

[aleksijohansson]

@jenshnielsen Even if I’ve built pyenv python as a framework matplotlib fails in an virtualenv built with pipenv. I’ve also understood that matplotlib works with python3 venv, but there is an issue open to get pipenv to work with python3 venv which is problematic for all kinds of weird reasons. If this is fixable in virtualenv that would be really good.

[jenshnielsen]

I know, I just tried to help you by telling you that it works with pyenv virtualenv which you can use to install create multiple pyenv envs with the same python version, since you claimed that pyenv does not handle that

[aleksijohansson]

@jenshnielsen Thanks for the tip, you are right that pyenv virtualenv provides a way to have multiple instances of the same version of python with pyenv and automatically switching between them per project. It also might be possible to install dependency modules from Pipfile into the pyenv virtualenv with pipenv using the flag to install to global python (pyenv virtualenv in this case). Since you said previously that this combo works fine, it can be a viable option in the scenario I was thinking about (using ”only” pyenv).

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.