Skip to content
This repository

don't close figures every cycle with inline matplotlib backend #892

Merged
merged 2 commits into from over 2 years ago

2 participants

Min RK Fernando Perez
Min RK
Owner

See #638

This allows gcf(), getfigs(), etc. to be useful again in the qtconsole and notebook.

draw_if_interactive() signals that the current figure is to be drawn.

Figures that are created and closed in the same cell will not be drawn.

I would like some testing of this, to see cases where it doesn't do as well as the current closing-everything model.

Min RK don't close figures every cycle in inline backend
This allows gcf(), getfigs(), etc. to be useful again in the qtconsole and notebook.

draw_if_interactive() signals that the current figure is to be drawn.

Figures that are created and closed in the same cell will not be drawn.

See #638
6c22e3f
Fernando Perez
Owner

The problem with this pattern is that it forces manual figure closes every time. Try in a qt console this:

plot(rand(100))
plot([1,2,3])

but do them one at a time, so the first happens in a single cell. The 2nd one will plot on top of the first.

forcing figure closes after every plot is pretty annoying, which is what led me to the design we have now. What I do if I want to keep using a figure is simply keep a reference to it:

# first cell
f = figure()
plot(rand(100))
# 2nd cell
plot([1,2,3])
# 3rd cell
f

I'm happy to revisit this, but I don't think this PR really solves the issue yet. Preserving the quick and easy flow we have now is paramount, right now the price to pay is having to keep references manually to figures you want to reuse. But being forced to issue a close on every plot call is definitely not the way to go: that's how the backend worked at the very start, and it would drive anyone crazy in real-world use.

Min RK
Owner

do them one at a time, so the first happens in a single cell. The 2nd one will plot on top of the first.

Yes, this is exactly the expected behavior, which is why this behavior has been reported as a bug. It is not logical to expect splitting a sequence of commands among IPython inputs to result in different behavior. This PR restores the expected matplotlib behavior of using figure() to create new figures.

It's also a potential problem that saving a session / exporting a notebook to Python will result in a different number of figures when run from outside IPython, or even re-running it in the exact same environment with any non-inline backend.

This isn't a big deal for people who never/always use inline, but for people who switch (me), it's a pretty big problem.

Preserving the quick and easy flow we have now is paramount

For my typical usage, this actually is a dramatic improvement in the 'quick and easy flow' of working with plots in IPython. In much of my interactive plotting, I do a lot of tweaking legends/titles/legends/axes. This is all very convenient to have in the toplevel namespace with title()/xlim()/etc., all of which are unavailable when IPython closes the figure inappropriately. Maintaining references to the figure doesn't help that at all.

I do the same reference-tracking things you mention when I work with the qtconsole, but it's a pretty significant step backward for interactive use, when terminal IPython is actually better than the qtconsole, and running the same code in the two produces different results.

right now the price to pay is having to keep references manually to figures you want to reuse.

it's more than that, because having to use references also means having to use object methods, rendering much of the pylab/pyplot functions useless outside the creating cell.

But being forced to issue a close on every plot call is definitely not the way to go: that's how the backend worked at the very start, and it would drive anyone crazy in real-world use.

This isn't accurate. It forces you to call figure() to create new figures, just like in every other matplotlib context, including every other IPython context. What it does is let IPython be internally consistent, rather than having the inline backend produce completely different figures from regular matplotlib backends.

I'm not saying that this is necessarily the right answer, just explaining my use cases, and why this definitely does improve them significantly.

Fernando Perez
Owner

I see your points and they have merit, but there's another side of this that worries me: if we don't close the figures for the user right away, then a long-running session with lots of plots will appear to leak memory like a sieve. That doesn't happen at the normal ipython because you see figures on your desktop, and tend to call close(all) regularly to clean up (if nothing else, to get them out of the way). But since here they are not visible anywhere, I bet users will end up with a ton of hidden figures holding references to arrays and other large data structures, likely to create memory pressure in long-running sessions.

Further, I actually find annoying the need to call figure() on new plots because in my mind, the inline backend just encourages a different workflow, and one that I think is natural. I realize it's different from the others, but I don't find it worse, actually I like it a lot: each cell (notebook or qtconsole) is like a 'little script' where I build my figures in full, setting labels, legends, etc. Then they get rendered immediately, and the slate is clean for the next figure. I find the reappearance of figures that are hidden out of the visual space jarring and annoying.

But I can see how someone can prefer an alternate workflow, you make a compelling case. This is a good candidate for making it an option: we both find fundamentally different worfklows more 'natural', so we're not likely to agree that the other is 'better' for either of us. Chances are, there will be a ton of people out there landing on either side of the question too, so we might as well make it an option, and in fact one where the default can be set in the user config but also that can be toggled at runtime with a magic. We probably should have a single magic to set options in this backend, such as png/svg and close behavior. This would reduce the proliferation of magics...

How does this sound?

Min RK
Owner
Fernando Perez
Owner

You're right on the refs held by the line collections and other mpl objects. That stuff probably holds on to the entire figure somehow, the mpl object trees are really crazy spaghetti salads and everything has a pointer to everything else.

Fernando Perez
Owner

Plan sounds good, if you can make it switchable it will actually be a big usability win for many people, I'm sure. Thanks!

Min RK add InlineBackendConfig.close_figures configurable
Allows switching between closing figures at the end of each cell, and
preserving active figures like other backends.
92666aa
Min RK
Owner

added configurable, keeping the same behavior as default. I'll explore the idea of a %config magic.

Min RK
Owner

The trick with %config is that changing the config object doesn't matter after everything is instantiated, so we would have to keep track of how to get the relevant instance of every configurable object.

Fernando Perez
Owner

Oh, I wasn't thinking of a generic %config magic for all configurables: just a magic to control the various parts of the pylab support. In fact we already have %pylab that can activate a backend at the cmd line, but it doesn't work in the nb. Perhaps the best path forward would be to:

  • fix %pylab so one can activate pylab support after startup also in the qtconsole and nb. That would bring them to feature parity with the terminal in that regard.

  • extend %pylab to allow changing at runtime various pieces of the inline backend behavior, such as image format and close behavior.

How does this sound?

Min RK
Owner

Ah, that's a much more reasonable goal. One problem, though: GUI support is implemented at the Kernel class level. We would have to be able to delete the running Kernel and create a new one with the right class in order to support activating a GUI at runtime, but I'm not quite sure how to do that.

Fernando Perez
Owner

Interesting! I replied this morning by email, but that message is not here (though I have it in my sent mail on gmail...). Anyway, pasting back my reply:

But at the terminal, we can already turn gui support on at runtime, so
I think the pieces are already there. Try %pylab at a normal
ipython session to test it. I think we just have an incorrect check
somewhere that is stopping this from working, because if it can work
at the terminal, the same thing should be possible to enable at any
other kernel.

Min RK
Owner

It's not a fundamental limitation, it's just made inconvenient by how we have written GUI support in the kernel. The GUI integration determines the Kernel subclass we use to start everything. Changing the gui at runtime means tearing down a Kernel, and bringing up a new one, without throwing away any of the Session/Shell/etc. objects that must be migrated.

Fernando Perez
Owner

Ah, I see. It's funny: back in the 0.10 series, that's how gui support worked for the shell, and our famous Shell.py file had a bunch of classes one for each gui type. When we built the inputhook support, we went away from that model, making it possible to toggle gui support at runtime. It's kind of ironic that the new code inherited that older design :)

I don't necessarily want to hold this PR forever on account of a convenience magic: should we just merge it as-is, and just leave an open issue on the larger refactoring of runtime control of the GUI in full kernels? Because that seems to be a larger problem that deserves its own effort, and that will let us close this guy...

Min RK
Owner

Sure, let's merge, and we can look at that later.

Of course, probably the better way is to make the eventloop stuff just functions, so they can be swapped out. The subclasses really only provide one method, so it shouldn't be much work to just replace the Kernel mapping with a loop-function mapping.

Fernando Perez fperez merged commit d2b3a0c into from October 18, 2011
Fernando Perez fperez closed this October 18, 2011
Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

Oct 17, 2011
Min RK don't close figures every cycle in inline backend
This allows gcf(), getfigs(), etc. to be useful again in the qtconsole and notebook.

draw_if_interactive() signals that the current figure is to be drawn.

Figures that are created and closed in the same cell will not be drawn.

See #638
6c22e3f
Oct 18, 2011
Min RK add InlineBackendConfig.close_figures configurable
Allows switching between closing figures at the end of each cell, and
preserving active figures like other backends.
92666aa
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 58 additions and 11 deletions. Show diff stats Hide diff stats

  1. 69  IPython/zmq/pylab/backend_inline.py
69  IPython/zmq/pylab/backend_inline.py
@@ -17,7 +17,7 @@
17 17
 from IPython.config.configurable import SingletonConfigurable
18 18
 from IPython.core.displaypub import publish_display_data
19 19
 from IPython.lib.pylabtools import print_figure, select_figure_format
20  
-from IPython.utils.traitlets import Dict, Instance, CaselessStrEnum
  20
+from IPython.utils.traitlets import Dict, Instance, CaselessStrEnum, CBool
21 21
 #-----------------------------------------------------------------------------
22 22
 # Configurable for inline backend options
23 23
 #-----------------------------------------------------------------------------
@@ -47,6 +47,23 @@ def _figure_format_changed(self, name, old, new):
47 47
             return
48 48
         else:
49 49
             select_figure_format(self.shell, new)
  50
+    
  51
+    close_figures = CBool(True, config=True,
  52
+        help="""Close all figures at the end of each cell.
  53
+        
  54
+        When True, ensures that each cell starts with no active figures, but it
  55
+        also means that one must keep track of references in order to edit or
  56
+        redraw figures in subsequent cells. This mode is ideal for the notebook,
  57
+        where residual plots from other cells might be surprising.
  58
+        
  59
+        When False, one must call figure() to create new figures. This means
  60
+        that gcf() and getfigs() can reference figures created in other cells,
  61
+        and the active figure can continue to be edited with pylab/pyplot
  62
+        methods that reference the current active figure. This mode facilitates
  63
+        iterative editing of figures, and behaves most consistently with
  64
+        other matplotlib backends, but figure barriers between cells must
  65
+        be explicit.
  66
+        """)
50 67
 
51 68
     shell = Instance('IPython.core.interactiveshell.InteractiveShellABC')
52 69
 
@@ -55,44 +72,74 @@ def _figure_format_changed(self, name, old, new):
55 72
 # Functions
56 73
 #-----------------------------------------------------------------------------
57 74
 
58  
-def show(close=True):
59  
-    """Show all figures as SVG payloads sent to the IPython clients.
  75
+def show(close=None):
  76
+    """Show all figures as SVG/PNG payloads sent to the IPython clients.
60 77
 
61 78
     Parameters
62 79
     ----------
63 80
     close : bool, optional
64 81
       If true, a ``plt.close('all')`` call is automatically issued after
65  
-      sending all the SVG figures. If this is set, the figures will entirely
  82
+      sending all the figures. If this is set, the figures will entirely
66 83
       removed from the internal list of figures.
67 84
     """
  85
+    if close is None:
  86
+        close = InlineBackendConfig.instance().close_figures
68 87
     for figure_manager in Gcf.get_all_fig_managers():
69 88
         send_figure(figure_manager.canvas.figure)
70 89
     if close:
71 90
         matplotlib.pyplot.close('all')
  91
+    show._to_draw = []
  92
+
72 93
 
73 94
 
74 95
 # This flag will be reset by draw_if_interactive when called
75 96
 show._draw_called = False
  97
+# list of figures to draw when flush_figures is called
  98
+show._to_draw = []
76 99
 
77 100
 
78 101
 def draw_if_interactive():
79 102
     """
80 103
     Is called after every pylab drawing command
81 104
     """
82  
-    # We simply flag we were called and otherwise do nothing.  At the end of
83  
-    # the code execution, a separate call to show_close() will act upon this.
  105
+    # signal that the current active figure should be sent at the end of execution.
  106
+    # Also sets the _draw_called flag, signaling that there will be something to send.
  107
+    # At the end of the code execution, a separate call to flush_figures()
  108
+    # will act upon these values
  109
+    
  110
+    fig = Gcf.get_active().canvas.figure
  111
+    
  112
+    # ensure current figure will be drawn, and each subsequent call
  113
+    # of draw_if_interactive() moves the active figure to ensure it is
  114
+    # drawn last
  115
+    try:
  116
+        show._to_draw.remove(fig)
  117
+    except ValueError:
  118
+        # ensure it only appears in the draw list once
  119
+        pass
  120
+    show._to_draw.append(fig)
84 121
     show._draw_called = True
85 122
 
86  
-
87 123
 def flush_figures():
88  
-    """Call show, close all open figures, sending all figure images.
  124
+    """Send all figures that changed
89 125
 
90 126
     This is meant to be called automatically and will call show() if, during
91 127
     prior code execution, there had been any calls to draw_if_interactive.
92 128
     """
93  
-    if show._draw_called:
94  
-        show()
95  
-        show._draw_called = False
  129
+    if not show._draw_called:
  130
+        return
  131
+    
  132
+    if InlineBackendConfig.instance().close_figures:
  133
+        # ignore the tracking, just draw and close all figures
  134
+        return show(True)
  135
+    
  136
+    # exclude any figures that were closed:
  137
+    active = set([fm.canvas.figure for fm in Gcf.get_all_fig_managers()])
  138
+    for fig in [ fig for fig in show._to_draw if fig in active ]:
  139
+        send_figure(fig)
  140
+    # clear flags for next round
  141
+    show._to_draw = []
  142
+    show._draw_called = False
96 143
 
97 144
 
98 145
 def send_figure(fig):
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.