Matplotlib warning about too many figures #2017

Closed
kshedden opened this Issue Sep 27, 2014 · 5 comments

Projects

None yet

2 participants

@kshedden
Contributor

There are apparently some issues in matplotlib with resource leaks in figures, which (apparently) are not always reference counted and garbage collected. Newer versions of matplotlib generate warnings when more than 20 figures are created.

This is a (minor) issue for me as I often generate large pdf files containing 100's of plots. I don't suppose there is a real figure resource limitation when writing to pdf, but I still get the warnings (I've never had any actual bad things happen, just warnings).

For statsmodels, we have the create_mpl_ax function that creates a new figure (if needed) using pyplot.figure. Supposedly if matplotlib.figure.Figure is called directly (bypassing pyplot), the reference counting and garbage collection work as expected:

http://stackoverflow.com/questions/16334588/create-a-figure-that-is-reference-counted/16337909#16337909

There is a lot of discussion about this topic on the matplotlib lists, and I haven't spent enough time to really understand it. It's not a major issue at this point but worth exploring eventually.

@josef-pkt
Member

I think we always call close at the end of a unit test, so there shouldn't remain any figures open that are not garbage collected, AFAIU.

For our plot funtions: Users have the choice to use the ax argument if they want to avoid the call to plt.figure. Also, user have a reference to the figure instances and can always close them when they are not needed any more.

From this, I don't see anything that we could/should do. Unless I'm missing something.

@kshedden
Contributor

I should look into it more before speculating too much, but based on what
people are saying, if the figure is created using matplotlib.figure.Figure,
then it will be garbage collected automatically when it goes out of scope,
but if it is created with pyplot.figure it will not be. If this is indeed
the case, then it would be easy to change create_mpl_ax to bypass pyplot
when creating figures. That way, people like me who create figures inside
of functions and forget to close the figure before the function exits won't
have this problem. It seems to only be an issue when generating many plots
programmatically (e.g. writing to pdf). For the pdf backend, there is no
actual figure window to close, and even though you save the figure (which
logically would finalize the figure object and close it), there is still a
reference to the figure inside pyplot so it will not be garbage collected
when the function exits. Recent matplotlib's have a warning about this
that I had not seen before.

On Sat, Sep 27, 2014 at 4:33 PM, Josef Perktold notifications@github.com
wrote:

I think we always call close at the end of a unit test, so there shouldn't
remain any figures open that are not garbage collected, AFAIU.

For our plot funtions: Users have the choice to use the ax argument if
they want to avoid the call to plt.figure. Also, user have a reference to
the figure instances and can always close them when they are not needed
any more.

From this, I don't see anything that we could/should do. Unless I'm
missing something.


Reply to this email directly or view it on GitHub
#2017 (comment)
.

@josef-pkt
Member

I'm not sure how we can do this for the users backend

in a plain interpreter session (spyder) with QTAgg backend:

>>> from matplotlib import figure
   fig = figure.Figure()
>>> fig = figure.Figure()
>>> ax = fig.add_subplot(1,1,1)
>>> r = ax.plot(np.linspace(0, 4*np.pi), np.sin(np.linspace(0, 4*np.pi)))
>>> fig.show()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Programs\Python27\lib\site-packages\matplotlib\figure.py", line 360, in show
    "created by pyplot.figure()." % err)
AttributeError: 'NoneType' object has no attribute 'manager'
Figure.show works only for figures managed by pyplot, normally created by pyplot.figure().

From Joe KIngton's answer it looks like we need the backend specific FigureCanvas.

And then I don't know how to actually show it.
fig.savefig('temp.png') works

It still looks to me this is a case for using the ax argument where the user knows what she wants, unless we add options for display versus save options.

@kshedden
Contributor

I think you are right. Thanks for explaining. I'm closing this issue.

@kshedden kshedden closed this Sep 28, 2014
@josef-pkt
Member

I checked that our plots that create a figure with several ax take a user given fig. It looks like we do this consistently.

So the main thing to make this easier, would be to add a utility function that creates a fig and, if necessary, an ax instance with a figure and canvas that doesn't use te Figure manager from pyplot.
That would be useful if the usage is only to save the figure without looking at it interactively (no show).
The main advantage of a helper function (even if it's just 3 lines of code) would be that we don't need to remember or know how to do this, and search on the internet each time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment