Stuck figure window after plt.close() in ipython with Qt5

[felixchenier]

Bug report

Bug summary

When more than one figure is shown, then clicking on the last figure's close button or running plt.close() or plt.close('all') leaves a zombie figure behind. At first, figure windows are properly closed, but when switching back to the matplotlib python application, the last figure resuscitates and is not clickable nor closable anymore.

Doing the samething another time leaves an additional zombie figure window, and so on. The only way to close those non-responding windows is to quit ipython.

Code for reproduction

Running in ipython:

%matplotlib Qt5
import matplotlib.pyplot as plt
plt.figure()
plt.figure()
plt.close('all')

Actual outcome

No output, and the figures close. But then if we press alt-tab, the last figure resuscitates and does not respond to user input.

Expected outcome

The last figure should not resuscitate.

Matplotlib version

• Operating system: macOS 10.15.4
• Matplotlib version: 3.1.3
• Matplotlib backend (Qt5Agg):
• Python version: 3.7.7
• Other libraries:

Installation under conda, default conda channel:

conda create -n test python=3.7
conda activate test
conda install ipython matplotlib pyqt

[TAC edited to remove comments in example code to make copy-pasting easier]

[tacaswell]

If you create another new figure do the existing ones become responsive again? Alternatively dose plt.show(block=True) make it responsive again?

My suspicion is that with all the windows closed, we are no longer spinning the QT event loop so while the window exists in the display manager side, nothing in handling the input.

[felixchenier]

@tacaswell
Thanks for your answer.

If you create another new figure do the existing ones become responsive again? Alternatively dose plt.show(block=True) make it responsive again?

Unfortunately no, the "closed" figure stays there and still does not respond.

My suspicion is that with all the windows closed, we are no longer spinning the QT event loop so while the window exists in the display manager side, nothing in handling the input.

I also had this suspicion, I tried many combinations but what I see is that anytime I created more than one figure, this problem happens some way or the other and it's impossible do close those stuck figures.

[felixchenier]

I realized that I was not running the latest stable version, which is 3.2.1. I installed it from conda-forge and I experience the same bug.

[QuLogic]

Throwing a plt.pause(0.1) after does not fix it, nor does opening a new figure or calling plt.close('all') again. It seems like this window is completely detached from Matplotlib at some point, and Qt is just running it because it wants to.

This is also a very old bug, which reproduced back to 2.2.5. Possibly it's something to do with IPython versions and its Qt integration.

[QuLogic]

This is also not macOS-specific.

[tacaswell]

I suspect that there is some race going on between IPython and Matplotlib trying to start up the global QApplication instance (but that is more of a hunch than anything solid).

[felixchenier]

I think I found a workaround. I manually connected the qt window destroy function to the figures' close events. My first tests seem to work, either using the pyplot close function, or the window close button (both cases had problems). Maybe it could serve as a direction to solve this bug in the real code.

%matplotlib qt
import matplotlib.pyplot as plt
                                                      
fig1 = plt.figure() 
fig1.canvas.mpl_connect('close_event', lambda _: fig1.canvas.manager.window.destroy()) 

fig2 = plt.figure() 
fig2.canvas.mpl_connect('close_event', lambda _: fig2.canvas.manager.window.destroy()) 

plt.close('all')

[felixchenier]

Interestingly, the workaround doesn't work anymore as soon as there's a plt.pause while the figures are shown.

Works:

%matplotlib qt
import matplotlib.pyplot as plt
                                                      
fig1 = plt.figure() 
fig1.canvas.mpl_connect('close_event', lambda _: fig1.canvas.manager.window.destroy()) 

fig2 = plt.figure() 
fig2.canvas.mpl_connect('close_event', lambda _: fig2.canvas.manager.window.destroy()) 

plt.close('all')

Doesn't work:

%matplotlib qt
import matplotlib.pyplot as plt
                                                      
fig1 = plt.figure() 
fig1.canvas.mpl_connect('close_event', lambda _: fig1.canvas.manager.window.destroy()) 

fig2 = plt.figure() 
fig2.canvas.mpl_connect('close_event', lambda _: fig2.canvas.manager.window.destroy()) 

plt.pause(0.1)
plt.close('all')

There seems to effectively has something with the event loop (I messed a bit with MPL's code and it's the start_event_loop from plt.pause that seems to cause this difference).

[tfetherolf]

I want to add that I encounter what seems to be a related issue when creating many figures in a script that is run from the command line. While the script does not hang on figures when plt.close() is used, creating a large number of figures in a loop shows that they are not properly closed when the open file limit is reached.

OSError: [Errno 24] Too many open files: 'figurename255.png'

Increasing or removing the limit on the number of open files allowed is not an adequate solution for me since this would cause extreme CPU slowdowns from the large number of figures I am creating.

Versions

Operating system: macOS 10.15.7
Python version: 3.8.5
Matplotlib version: 3.3.1
Matplotlib backend: MacOSX
Spyder version: 4.1.5
Qt version: 5.9.7
PyQt5 version: 5.9.2
iPython version: 7.18.1

Code for Reproduction

Running as a script from command line.

import matplotlib.pyplot as plt

for i in range(1000):
    fig = plt.figure()
    plt.savefig('figurename'+str(i)+'.png')
    plt.close()

Adding fig.canvas.mpl_connect('close_event', lambda _: fig.canvas.manager.window.destroy()) after fig = plt.figure() does not prevent the script problem for me, but does work for closing windows when using the iPython console in Spyder.

Current Workaround

Using plt.clf() instead of opening/closing many figures is an adequate workaround in my case.

import matplotlib.pyplot as plt

fig = plt.figure()
for i in range(1000):
    plt.savefig('figurename.png')
    plt.clf()

[jklymak]

@tfetherolf, this seems somewhat different since its from the command line and never uses an interactive backend, can you open a new issue for this?

[tacaswell]

@tfetherolf If you do matplotlib.use('agg') before creating your first figure does it fix the problem?

[tacaswell]

In both cases my new suspicion is that for various reasons we defer setting up the logic to tear down the GUI windows until after the GUI window is shown, however if you close the figure before it is shown we end up in a state where on one hand we have asked a GUI to be shown, then tear down the Python side logic, then the window shows and we get a zombie window (with no Python side way to get a hold of it!) and no logic of when to close it.

[tfetherolf]

@tacaswell Adding matplotlib.use('agg') to the top of my script (before the first figure) does prevent the crash from happening.

[tacaswell]

doing a gc.collect() will clear the window so there is something stuck in garbage collection purgetory that is delaying tearing down the Python object (even if we can not get to it any more) which is in turn delaying tearing down the c++ object which means it is alive enough for Qt to try to show it but not alive enough to be usable.

This is connected to #18609 (comment) in the sense that we are not sure if "close" means "hide" or "destroy".

[felixchenier]

Hello @tacaswell
On my side the garbage collector doesn't clear the window (on macOS). Just so that you know...

[tacaswell]

Interesting, still suspect it is a fussy reference counting issue, but apparently a platform dependent one...

[QuLogic]

@tacaswell was this perhaps fixed by the garbage collection change?

[oscargus]

It seems like the original issue is gone. At least on Windows 11. But since I do not know if it existed on Windows, it is hard to say if it is fixed...

[tacaswell]

I can confirm that this is not a problem on linux, but it does still happen on osx (using qt5 from Conda-forge).

My current theories are:

• two Applications are being created
  • but I only get 1 extra window not two
  • ... more than 1 Application tends to result in segfaults
  • ... if this were the case we should be getting warnings / errors if we make a window and then use it
• qt and osx are getting out of sync
  • but betting on bugs in libraries of this scale is always a bad call!

I also notice if you use the dock to try and kill the Matplotlib logo app it does not actually close any of the windows (live ones or the zombies), you can not make any more, but python is fine.

If you then do plt.show(block=True) you can make windows again and everything seems happy again....

@QuLogic where else did you reproduce this?

[QuLogic]

Note that you need to do the steps from the first post in a single cell.

I can reproduce in my existing conda environment, with Python 3.9, Matplotlib main, and:

ipython                   8.8.0              pyh41d4057_0    conda-forge
pyqt                      5.15.7           py39h18e9c17_2    conda-forge
pyqt5-sip                 12.11.0          py39h5a03fae_2    conda-forge
qt-main                   5.15.6               h602db52_6    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge

I cannot reproduce with Fedora system packages:

$ rpm -q python3-qt5-base qt5-qtbase python3 python3-ipython python3-matplotlib
python3-qt5-base-5.15.6-10.fc37.x86_64
qt5-qtbase-5.15.8-5.fc37.x86_64
python3-3.11.1-3.fc37.x86_64
python3-ipython-8.5.0-1.fc37.noarch
python3-matplotlib-3.6.3-1.fc37.x86_64

[QuLogic]

I also went through all iterations of conda create -n foobar python=3.X ipython matplotlib pyqt for X=9:

  ipython            conda-forge/noarch::ipython-8.11.0-pyh41d4057_0
  matplotlib         conda-forge/linux-64::matplotlib-3.7.1-py39hf3d152e_0
  matplotlib-base    conda-forge/linux-64::matplotlib-base-3.7.1-py39he190548_0
  pyqt               conda-forge/linux-64::pyqt-5.15.7-py39h5c7b992_3
  pyqt5-sip          conda-forge/linux-64::pyqt5-sip-12.11.0-py39h227be39_3
  python             pkgs/main/linux-64::python-3.9.16-h7a1cb2a_1
  qt-main            conda-forge/linux-64::qt-main-5.15.6-hd477bba_1

for X =10:

  ipython            conda-forge/noarch::ipython-8.11.0-pyh41d4057_0
  matplotlib         conda-forge/linux-64::matplotlib-3.7.1-py310hff52083_0
  matplotlib-base    conda-forge/linux-64::matplotlib-base-3.7.1-py310he60537e_0
  pyqt               pkgs/main/linux-64::pyqt-5.15.7-py310h6a678d5_1
  pyqt5-sip          pkgs/main/linux-64::pyqt5-sip-12.11.0-py310h6a678d5_1
  python             pkgs/main/linux-64::python-3.10.9-h7a1cb2a_1
  qt-main            pkgs/main/linux-64::qt-main-5.15.2-h327a75a_7

and for X = 11:

  ipython            conda-forge/noarch::ipython-8.11.0-pyh41d4057_0
  matplotlib         conda-forge/linux-64::matplotlib-3.7.1-py311h38be061_0
  matplotlib-base    conda-forge/linux-64::matplotlib-base-3.7.1-py311h8597a09_0
  pyqt               conda-forge/linux-64::pyqt-5.15.7-py311ha74522f_3
  pyqt5-sip          conda-forge/linux-64::pyqt5-sip-12.11.0-py311hcafe171_3
  python             conda-forge/linux-64::python-3.11.0-h10a6764_1_cpython
  qt-main            conda-forge/linux-64::qt-main-5.15.6-hd477bba_1

Starting with the 3.10 environment, I could not reproduce the problem. I noticed that qt was coming from main there instead of conda-forge, but it went back to conda-forge with 3.11, and the problem is gone there as well.

I don't know what exactly would have fixed it, but it seems like upgrading everything will do?

[jcl-feritech]

To contradict tacaswell, this does seem to be a problem in linux, it has been happening to me for about 18 months now.
For me, I started seeing this when I upgraded from Ubuntu 18 to Ubuntu 22.
I am running in ipython3, python3 version 3.10.12. I can see this problem with just one window open for example:

plt.figure('hello world')
plt.plot(x, y)
plt.show(block=False)

Will leave the window hanging, not everytime, but often enough to be annoying. It closes when I close ipython3.

[tacaswell]

I could not reproduces this 5 times in a row with

✔ 12:28:43 [belanna] @ ipython
Python 3.11.8 (main, Feb 12 2024, 14:50:05) [GCC 13.2.1 20230801]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.22.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import matplotlib.pyplot as plt
   ...: plt.figure('hello world')
   ...: plt.plot([], [])
   ...: plt.show(block=False)

In [2]: plt.matplotlib.get_backend()
Out[2]: 'QtAgg'

When this does not work, what is the result of

In [3]: ip = get_ipython()

In [4]: ip.inputhook
Out[4]: <bound method TerminalInteractiveShell.inputhook of <IPython.terminal.interactiveshell.TerminalInteractiveShell object at 0x74df290d20d0>>

?

Does calling plt.ion() before you create the figure reliably fix things?

The core of the problem is that when you do plt.show(block=False) we are trusting that something else is managing the inputhook so that the GUI event loop spins while waiting for you to type again and we try to stay out of that (because configuring that is the responsibility of who/what ever is running your shell).

@jcl-feritech Can you also give use your mpl version, the exact IPython version, which qt binding you are using and it's exact version, and the version of Qt it is built against? Which window manager you are using and if it is X11 or wayland may also be relevant.

[jcl-feritech]

I am using ipython version 7.31.1, mpl version 3.5.1.

Sadly plt.ion() does not fix things.
I am using Ubuntu 22.04.2 LTS, 64 Bit.
Gnome version 42.5
Windowing System Wayland.

I have realised that I am using TkAgg, not Qt, sorry for the confusion.

[tacaswell]

Do you have the same problem with tk in mpl3.8.3 ?

[jcl-feritech]

I am testing mpl3.8.3, and I can't reproduce the problem yet, I will report back here if it breaks in due course. Thank you.