In 3.0.0 backend cannot be set if 'get_backend()' is run first

[moonshoes87]

Bug report

Bug summary

In my code, I need to check whether the matplotlib backend has already been set. If it has not been set, I need to then set it to 'WXAgg'. In matplotlib 2.*, I was able to do this, but in matplotlib 3.0.0 it fails.

Code for reproduction

import matplotlib
if not matplotlib.get_backend() == 'WXAgg':
    matplotlib.use('WXAgg')
print('backend:', matplotlib.get_backend())

Actual outcome

With matplotlib 3.0.0:

/Users/nebula/Python/PmagPy/programs/test.py:7: UserWarning: matplotlib.pyplot as already been /Users/nebula/Python/PmagPy/programs/test.py:7: UserWarning: matplotlib.pyplot as already been imported, this call will have no effect.
  matplotlib.use('WXAgg')
backend: MacOSX

Expected outcome

With matplotlib 2.2.3:

backend: WXAgg

Matplotlib version

• Operating system: OSX High Sierra 10.13.6
• Matplotlib version: 3.0.0
• Matplotlib backend (print(matplotlib.get_backend())):
• Python version: 3.6.6 (Anaconda, default channel)
• Jupyter version (if applicable):
• Other libraries:

[jklymak]

I can't reproduce. Are you doing this from inside an interactive shell? Just running the above as a script I get the expected outcome on both master and v3.0.0

[moonshoes87]

The same result occurs when I run the code within the Python interpreter or as a script.

One difference is that if I run it as an executable: ./test.py the backend is MacOSX, whereas if I run it as a python program: python test.py, the backend is Qt5Agg. But in either case, I get the same user warning.

Here is the full matplotlib version info:

matplotlib                3.0.0            py36h54f8f79_0 

[ImportanceOfBeingErnest]

I can reproduce this with matplotlib 3.0.0

It's pretty strange that the warning complains about pyplot being imported without it being imported anywhere. Here the mpl300 environment has been completely created via conda, i.e. conda create -n mpl300 matplotlib=3.0.

If using master, the problem does not occur.

In both cases, matplotlib.get_backend() is 'Qt5Agg'.

Also when creating a conda environment without matplotlib, conda create -n mpl300pip numpy, and then installing matplotlib via pip,

> activate mpl300pip
(mpl300pip) > python -mpip install matplotlib==3.0

and then running the code, the warning does not occur:

Does conda do something to the matplotlib package, which it shouldn't? The matplotlib.__init__ files are identical between the conda-installed and the pip-installed matplotlib. What else could be different? Other than the python version in use? Should that matter?

[anntzer]

Perhaps conda still ships a matplotlibrc with a backend: set? dunno.

[ImportanceOfBeingErnest]

Ok let appart conda, how can it be that if you import matplotlib you get a warning about pyplot being imported? That should never happen, should it?

So what's the purpose of this line?

matplotlib/lib/matplotlib/__init__.py

Lines 876 to 880 in d5a4eda

	elif key == "backend":
	val = dict.__getitem__(self, key)
	if val is rcsetup._auto_backend_sentinel:
	from matplotlib import pyplot as plt
	plt.switch_backend(rcsetup._auto_backend_sentinel)

[Kaushalya]

This line throws me the following error if I import matplotlib inside PyCharm.
AttributeError: module 'matplotlib.pyplot' has no attribute 'switch_backend'
Works fine on the interpreter.

[ImportanceOfBeingErnest]

@Kaushalya This sounds like a different issue, especially if it's working correctly in the command line. Note that PyCharm does a lot of crazy stuff with matplotlib internally (seen from all the results in their source code) so it may also be an issue with PyCharm only. If you think this to be a matplotlib issue (i.e. if you can reproduce this outside of PyCharm), I would suggest to open a new issue about it, including all details and full traceback.

[moonshoes87]

Any updates on this issue? Or is this something that should be raised with the Anaconda people instead?

[ImportanceOfBeingErnest]

Given that the issue does not occur when matplotlib is installed via pip suggests at least that conda might know better what's happening.

[tacaswell]

@moonshoes87 The issue is that we moved to an run-time fallback mechanism to sort resolve what GUI framework to use. Internally we set the backend to a sentinal to indicate that the fall-back / discovery code needs to run, however we do not want to return that to the users (for reasons of type stability) or to return a placeholder string (as it is not a real backend) so when you call mpl.get_backend() the discovery code gets run and in your case seems to discover the osx backend. Your next line that says "you can not change GUI frameworks, you stuck with OSX".

Given the other constraints, I am not sure we can fix this for you. I suggest that you either setup a .matplotlibrc file to select wx (which will fix the behavior in code you can not change with new versions of Matplotlib) or remove the if... check in code you can change.

I am a bit confused about why this is only a problem in anaconda (them shipping a .matplotlibrc seems correct, but then get_backend should not run through the fallback mechanism), but both of the above should work there.

[jklymak]

@tacaswell get_backend() just says "Return the name of the current backend" and I think its reasonable for users to not expect that to do anything, like import pyplot and set the backend.

I don't see how this is conda's fault. I have no idea what matplotlibrc they ship, but I should be able to run the above code w/ no matplotlibrc.

[anntzer]

The correct fix IMO is what I proposed in #11600 (comment), which is to make use() succeed even if the backend has already been "set", as long as the event loop did not start yet.
This is what some early versions of the backend switching PRs did, but that got scrapped due to disagreement over the API.

[tacaswell]

@tacaswell get_backend() just says "Return the name of the current backend" and I think its reasonable for users to not expect that to do anything, like import pyplot and set the backend.

However we can not know the 'current backend' until we do the fallback process.

succeed even if the backend has already been "set", as long as the event loop did not start yet.

Do we have good handle on when each of the event loops gets started / checking it is started? I think we need to be cautious or more than just if the loop is running, but if input_hook has been set up.

[anntzer]

Do we have good handle on when each of the event loops gets started / checking it is started? I think we need to be cautious or more than just if the loop is running, but if input_hook has been set up.

We have get_running_interactive_framework() which is supposed to check just that.

[tacaswell]

I don't think we defer the imports long enough, if you have the rcparams set to Qt5, then even though the event loop will not have started yet you will not be able to switch to Qt4.

In principle I agree it sounds good, but I am worried about it only working sometimes.

[anntzer]

In that specific case you'll get an ImportError, which is just fine with me.
Remember that the current behavior of use() is that if pyplot is already imported, then we won't switch, except if force=True, in which case, well, we'll give it a try but it may or may not work. (And on top of that we may or may not get a warning depending on the value of warn.)
The proposed behavior is that neither force nor warn will exist anymore, and either the switch will silently succeed, or raise an ImportError. If you are really worried about breaking some setups we can always keep force with a default of True, with the meaning "silently suppress ImportError"...