matplotlib 3 pyplot on MacOS bounces rocket icon in dock

[defjaf]

Bug report

Bug summary

Rocket icon appears in dock (and bounces) when matplotlib.pyplot is imported after installing matplotlib 3.0. In previous versions, this didn't happen (and still doesn't with python2 on the same machine)

Code for reproduction

import matplotlib.pyplot 

Actual outcome
Rocket icon appears in MacOS dock and bounces for a while.

Expected outcome
Nothing (no dock icon for previous versions).

Matplotlib version

• Operating system: MacOS
• Matplotlib version: 3
• Matplotlib backend: any (seems to happen no matter what is set in matplotlibrc)
• Python version: 3.7 (Python.org framework build)

Installed from pip.

I haven't seen any other complaints about this, so I suspect that there is something specific to my setup.

Perhaps related to this ipython issue (but this happens even from the plain python3 REPL.) I

[tacaswell]

Is python hung while it is bouncing? If so I suspect that this is related to the font search issue (see #12176 / #12177 ). We have a fix in, and there will likely be a 3.0.1 in a few weeks. For now I suggest either going to an empty directory, doing the import there to let the font cache build.

[defjaf]

No, I don't think it's that. Python and matplotlib are running just fine with the icon in the dock.

(Aside: I do think I've separately encountered the font search issue, however -- when diagnosing this, I tried to remove my .matplotlib directory and definitely found several issues with the system trying to set up fonts: 1. it stops whenever it finds a bad symlink anywhere in my filesystem; 2. Even after I fixed all of those, it still eventually failed, after more than 20 minutes, with FileNotFoundError: [Errno 2] No such file or directory: '/Users/jaffe/.matplotlib/fontlist-v300.json')

[defjaf]

Perhaps I should ask if anyone else on MacOS is (or isn't) seeing this behaviour?

[defjaf]

See also this post from Dr Drang regarding Python 2.7 in 2014, although I think the icon doesn't bounce in that case.

Note that the plist hack still works, but matplotlib.use("Agg") doesn't anymore. Could this be due to the fallback backend search?

But this brings up another question: I don't see this behaviour under Python 2.7. I can get a non-bouncing icon in the dock if I set to a GUI backend, but never a bouncing icon (and no icon if I have a non-GUI backend).

[gsportelli]

I have the same problem. The rocket icon starts bouncing after importing matplotlib.pyplot only if I upgrade from matplotlib==2.2.2 to matplotlib==3.0.0.

[jklymak]

This is somewhat confusing as Matplotlib 3.0.0 does not run under python 2.7. So did you mean Matplotlib 2.2.x doesn't bounce?

I have a Mac and I never see the rocket icon after I set my backend to Qt5Agg. Have you tried setting this in ~/.matplotlib/matplotlibrc? Its possible the new renderer fallback is not working properly...

This'll be hard for me to debug - I use conda for my install, not python.org + pip. If you go that route, its entirely possible there are mismatched libraries, and you should consider upgrading all the non-python dependencies...

Sorry to not be more helpful.

[defjaf]

To be clear:

• Python 3 + Matplotlib 3: bounces no matter what backend
• Python 2.7 + Matplotlib 2.2x: gives no icon if launched without a gui backend in matplotlibrc, fixed (not bouncing) icon if launched with gui backend (as in the Dr Drang post)
• Python 3 + Matplotlib 2.2x: I don’t have this setup on my machine right now, but I recall that it behaves as on Python 2.7.

[jklymak]

• Python 3 + Matplotlib 3: bounces no matter what backend

That is hard to understand, and my guess is that its always choosing the same backend, depsite your intentions

Can you put

import matplotlib
print(matplotlib.get_backend())

at the top of your script and check that its really giving the intended backend?

[defjaf]

Yep:

$ python3
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> matplotlib.get_backend()
'agg'
>>> import matplotlib.pyplot as plt

bounce - bounce - bounce

[gsportelli]

I can confirm that with Python 3 and Matplotlib 2.2.2 the icon does not show up.

[jklymak]

@defjaf Sorry, I can't reproduce conda py3.7, mpl3.0.0. I don't get any icons in the doc until I do plt.show(). Must be something funky w. the framework build.

[defjaf]

@jklymak So does that mean "wontfix" or do we need to dig further?

[jklymak]

Oh no absolutely not. It means I probably won’t fix, but hopefully someone on the dev team uses non-conda on a Mac.

Do you ever get the prompt back? Can you check backend after you get the prompt back?

[defjaf]

@jklymak Aha, I think there's a misunderstanding: the bouncing icon does not hang the interpreter. The prompt returns immediately and it runs just fine.

[jklymak]

@tacaswell @anntzer is it possible the new backend code is importing and checking a backend before accepting the OP's choice of "Agg"?

[tacaswell]

if you do get_backend() again after importing pyplot what do you get?

If you start a fresh interpreter can you do:

import matplotlib
dict.__getitem(matplotlib.rcParams, 'backend')

(which side steps the auto-fallback logic)

To check, backend: 'agg' is in your rcparams file? It make be better to test with import matplotlib; matplotlib.use('BACKEND_OF_CHOICE') just to be sure!

One possible concern is that the osx backend is importing just enough to trigger the rocket, but not enough to be selected?

It may also help to put some print statements in the matplotlib.switch_backend function to see when that is being called with what inputs.

@defjaf Thanks for sticking with us to sort this out!

[defjaf]

get_backend and the matplotlib.rcParams dict behave as expected (note that the syntax is somewhat different than you suggested):

$ python3
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib; matplotlib.rcParams.__getitem__('backend')
'agg'
>>> matplotlib.use('PDF'); matplotlib.rcParams.__getitem__('backend')
'pdf'
>>> import matplotlib.pyplot
>>>

bounce - bounce - bounce ...

(And indeed if I actually use matplotlib I don't get a GUI.)

Also, FYI, no difference in behaviour if I set backend_fallback: False in matplotlibrc.

[jklymak]

OK, my appologies - I can reproduce on my anaconda set up. I don't get a rocket, but I get an app icon that flashes for a while and a beachball if I don't change focus back to the terminal. Qt5Agg is the backend both before and after import matplotlib.pyplot as plt.

This is definitely to do with the Framework build - if I use the non-framework build, and use Qt5Agg, the extra dock icon never shows up.

The dock icon controls the visibility of the window that shows up if I call plt.show(). So, somehow importing pyplot is now queing up the figure manager (?) in the Framework Build but not in the non-framework build. Note again that Qt5Agg is my default backend.

Reverting to 2.2.3 this doesn't happen.

[tacaswell]

er, I meant dict.__getitem__(matplotlib.rcParams, 'backend') The goal is to side-step

matplotlib/lib/matplotlib/__init__.py

Lines 876 to 880 in 09e9504

	elif key == "backend":
	val = dict.__getitem__(self, key)
	if val is rcsetup._auto_backend_sentinel:
	from matplotlib import pyplot as plt
	plt.switch_backend(rcsetup._auto_backend_sentinel)

The bouncing does not start until you import pyplot though?

I suspect the issues are in

matplotlib/lib/matplotlib/pyplot.py

Lines 2331 to 2339 in 09e9504

	# If rcParams['backend_fallback'] is true, and an interactive backend is
	# requested, ignore rcParams['backend'] and force selection of a backend that
	# is compatible with the current running interactive framework.
	if (rcParams["backend_fallback"]
	and dict.__getitem__(rcParams, "backend") in _interactive_bk
	and _get_running_interactive_framework()):
	dict.__setitem__(rcParams, "backend", rcsetup._auto_backend_sentinel)
	# Set up the backend.
	switch_backend(rcParams["backend"])

[jklymak]

Just commenting all that out doesn't change the behaviour for me (and the if conditional is None)

[defjaf]

er, I meant dict.__getitem__(matplotlib.rcParams, 'backend')

Same (correct) behaviour as above.

The bouncing does not start until you import pyplot though?

Exactly.

I suspect the issues are in matplotlib/lib/matplotlib/pyplot.py
Lines 2331 to 2339 in 09e9504

OK, I've played with this code a bit.

The if (rcParams["backend_fallback"] block doesn't matter, but if I comment out switch_backend(rcParams["backend"]) I don't get a bouncing with a non-GUI backend. However, if I do matplotlib.use("MacOSX") and then import matplotlib.pyplot as plt, I get the bouncing icon.

But note that this behaviour is still a bug, I think -- under 2.7, the icon didn't bounce.

[defjaf]

Just commenting all that out doesn't change the behaviour for me (and the if conditional is None)

Hmmm: As per above, the if... block has no effect for me, but switch_backend does help, at least for non-GUI backends.

[jklymak]

Not surprisingly bisects to 56acb0f

git bisect bad
56acb0f691f3447f7f15b257b164e0bc431760eb is the first bad commit

[jklymak]

The culprit looks to be:

matplotlib/lib/matplotlib/pyplot.py

Lines 214 to 215 in 09e9504

	current_framework = \
	matplotlib.backends._get_running_interactive_framework()

If I run this line I get the bouncing dock - so I guess _get_running_interactive_framework checks for the right frame work, even if MacOSX is not the framework...

[jklymak]

#12207 removes the check that opens the framework if using a non-GUI backend. If you use a GUI backend, I guess this icon coming up upon import is expected behaviour (?)...

[tacaswell]

I think there are actually two issues here,

1. why the rocket is bouncing
2. why if the user has used mpl.use we are going through auto-discovery (see matplotlib 3 pyplot on MacOS bounces rocket icon in dock #12188 (comment))

[laurentperrinet]

I can confirm the same behavior on

Python : 3.6.5 64bit [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
OS : Darwin 18.0.0 x86_64 i386 64bit
matplotlib : 3.0.0

I had written a question on stackoverflow which is well described by this issue.

[jklymak]

OK, to clarify a few things:

Using the conda framework build (pythonw)

v2.2.3:

If I run:

>>> import matplotlib
>>> matplotlib.use('macosx')
>>> import matplotlib.pyplot as plt

Similarly:

>>> from matplotlib.backends import _macosx

The terminal loses focus and the icon "bounces" if I use the conda framework build.

Conclusion: the bouncing has always been there if you specify "macosx" and use the framework build. So @tacaswell's issue 1 above is not something we should consider fixing as a bug fix for 3.0.x. If someone wants to figure out this behaviour, they should feel welcome to do so.

As @dstansby points out above the origin of this behaviour is in the verify_framework() routine in _macosx.m. If you remove this check, then the icon is not activated until draw time.

v3.0.0

The issue is that with the new changes, import pyplot calls _get_running_interactive_framework, even if mpl.use has been called. _get_running_interactive_framework tries to import _macosx which checks that the framework build is being used, and pops up the icon to do so. It needs to do this because the way to tell if a macosx GUI is being used is to call _macosx.event_loop_is_running().

@anntzer or @tacaswell will have to explain the logic here, but I think the idea is that if a backend has not been set, we should check to see if Matplotlib is supposed to be running in GUI. But it seems to be checking even if the backend has been set, and that is what is causing the problem here.

I don't understand the new backend switching logic, but we either need to not call _get_running_interactive_framework or move _macosx.m/verify_framework() out of the init, so it doesn't get called when we import _macosx and call it somewhere else.

Assigning the folks who understand this logic better, but more than willing to help test changes...

[anntzer]

I'll explain the current design and why it is as it is.

Before matplotlib 3.0, we shipped (i.e., Matplotlib wheels, conda packages, linux packages, etc.) a default matplotlibrc file with a backend: ... entry. Depending on the build details, common values for that entry were be "agg" (not great for interactive use), "macosx" (only for OSX builds, doesn't work on non-framework builds), "tkagg" (works nearly everywhere although some linuxes distribute tkinter separately, and fails on headless systems), or "qt5agg" (done by conda but adds another (heavy!) dependency on PyQt).

Given the issues described above, for matplotlib 3.0, we changed the setup so that the default matplotlibrc does not include a backend: ... entry, and added the behavior below when this entry indeed doesn't exist. Note that if the end user (or third-party packager) provides their own matplotlibrc with a backend: ... entry, nothing changes.

If there is no backend: ... entry, try to import backends in a statically defined order, namely macosx, qt5agg, ..., ending with agg -- a process we termed "backend fallback". If a backend fails to load with an ImportError, move on to the next one. If a backend requires a GUI framework that is incompatible with the currently running GUI framework (if any) (see below), also skip it. The first non-skipped backend becomes the default backend. The result is that when running on an OSX framework build, we get the macosx backend; otherwise, if PyQt5 is installed, we get qt5agg; etc.; if running headless, we get agg. Ignoring implementation questions for now, this is indeed the desired behavior (see the numerous issues that previously existed regarding this topic).

How do we know whether "the backend requires a GUI framework that is incompatible with the currently running GUI framework (if any)"? Backends have been changed to additionally expose a required_framework attribute, that is a string representing the framework ("macosx", "qt5", etc.) or None (for GUI-less backends); conversely, there is a _get_running_interactive_framework function that checks what the currently running framework is (if any); we just compare the result of _get_running_interactive_framework() to backend_mod.required_framework.

However, it is clear that this comparison must occur during backend fallback. Therefore, moving the check down to Canvas or Manager instantiation time, as proposed by #12557, won't work, in the sense that backend fallback won't be able to detect some cases (namely, non-framework Pythons that don't have a backend: ... in their matplotlibrc) where the macosx backend shouldn't be loaded, and will load it incorrectly.

Really, the main necessity of the check is to know whether, yes or not, the current Python build is a framework Python build or not. I'd hope this is something that can be done without triggering rocket bounces, but I don't know enough about OSX to tell. It is somewhat sad, however, that the "official" way to perform that check (checking some stuff in the sysconfig module) doesn't work because conda's "framework" build is not a real framework build but something that fakes it :/ (

[defjaf]

An interesting point that may have not been noted before:

On my python.org framework setup, Under the python2 REPL

from matplotlib.backends import _macosx

does not bounce the icon -- the icon appears, but it doesn't bounce. But it does appear and bounce under ipython2 or jupyter with a python2 kernel.

[jklymak]

OK, but thats 2.3.3 behaviour. Lets try to stick w/ the changes in 3.0.0 for this issue, which is that the icon gets activated even when it should not.

If you think the old 2.2.3 behaviour is also undesirable, maybe open a new issue?

[defjaf]

I was pointing it out mostly in case the different REPL v ipython behaviour was a hint about something!

[dstansby]

Thanks for the info @jklymak and @anntzer. So, given the backend check needs to be done for the fallback logic, this is what I've found out about the bouncing icon. It is caused by one of these two lines in _macosx.m (inside the check for the framework build):

matplotlib/src/_macosx.m

Lines 2584 to 2585 in a43fd85

	&& GetCurrentProcess(&psn)==noErr
	&& SetFrontProcess(&psn)==noErr) return true;

Removing one or both of those checks stops the icon bouncing, but doing so does not properly identify framework vs. non-framework builds (#12252). Both of those calls are from the old OSX Carbon framework, and they were both deprecated in OSX 10.9 (a while ago!).

So, the answer might lie in replacing the calls to GetCurrentProcess and SetFrontProcess with more modern, non-deprecated calls to the Cocoa framework. Information on what I think is the relavent API can be found here: https://developer.apple.com/documentation/appkit/nsrunningapplication I have no idea how one would go about replacing the deprecated calls however, and some googling doesn't bring up much useful information.

Alternatively the call to SetFrontProcess might be causing the bouncing. Maybe if there is a way to un-set the front process (set it back to what it was before that call?) that would stop the bouncing.

[jklymak]

OK, what changed wrt to the rocket is that we took out the chunk of code that compiled if we were in 10.6 or greater in #11850. Thats whats changed from 2.2.3 and is causing the bouncing behaviour, as @dstansby points out above.

[jklymak]

@defjaf Are you able to test master? #12603 should fix the bouncing for most backends except "macosx", but it would be useful for more people to try.

[tacaswell]

This looks to be partially resolved (should only affect people who intend to use the macosx backend?) pushing to next milestone as we want to get 3.0.1 out the door.

[defjaf]

Hi @tacaswell and @jklymak: I have managed to get master running here, and it does seems to solve the problems as stated: the icon only shows up for the macOSX backend. Curiously, and perhaps related to previous comments, it bounces from the standard REPL, but the icon appears without bouncing in ipython or jupyter.

[tacaswell]

I wonder if this is somehow related to readline.....

[anntzer]

Alternatively the call to SetFrontProcess might be causing the bouncing. Maybe if there is a way to un-set the front process (set it back to what it was before that call?) that would stop the bouncing.

Indeed, perhaps it'll help to first call GetFrontProcess (https://developer.apple.com/documentation/applicationservices/1501050-getfrontprocess?language=objc) before doing all that manipulation, and restore it at the end?

[jklymak]

I'm going to close because I think this is as good as we are going to get for now. There is a PR for 3.1 (#12633) that might make life even better for mac backend users...

[nielskou]

I had this problem in matplotlib==3.0, upgrading to 3.1 solved my problem.

[neilyoung]

3.4.2 bounce bounce bounce :/

[jklymak]

@neilyoung please open a new issue with version info, and exactly how you installed Matplotlib and python. Thanks!

[neilyoung]

Thanks for the response. Meanwhile I found a work around by plt.pause(0.0001). For the files: plotlib is 3.4.2, installed globally under python 3.9 at macOS 11.4

[jklymak]

How did you install python 3.9, and what does "installed globally" mean?

[neilyoung]

Sorry for not being specific enough. 3.9 comes with brew, install globally means "sudo python3 -m pip install matplotlib==3.4.2"

EDIT: 3.9 is default with macOS 11

[jklymak]

I cannot reproduce with python3 3.9.1 from homebrew, neither by calling

python3 test.py

import matplotlib.pyplot as plt 

nor via the REPL. I did not install globally, because that is not a good idea in general, so maybe that is the difference?

Again, please open a new issue, and explicitly state what you did to install python and what you did to install matplotlib. Further state exactly what steps you took to get the bouncing doc.

[neilyoung]

This is not my app :)

I'm running a script in a "for" loop, which updates dots and lines and calls "plt.draw()". This script (not mine) is supposed to work on other OS.

Again. I'm through with this. For me the pause works.

Thanks. Don't worry