Qt5Agg blitting issue with matplotlib 2.2.2

[ericpre]

Bug report

Bug summary

Plotting is broken in hyperspy (1.3 and dev) following matplotlib 2.2.2 release: the animated artists are not drawn as expected. In the dev version of hyperspy, the recursive repaint issue (canvas.blit call in a draw_event callback) has been fixed.
We used blitting and the plotting code is quite convolved (a bit messy) in hyperspy, I have been looking for the bug in hyperspy -typically not blitting correctly- but I couldn't find any mistake...

Code for reproduction

This is a minimal example to reproduce the issue without using hyperspy. This is designed to be run in interactive environment (ipython).

%matplotlib qt
import numpy as np
import matplotlib.pyplot as plt

class BlittedFigure(object):

    def __init__(self):
        self._background = None

    def _set_background(self):
        if self.figure:
            canvas = self.figure.canvas
            if canvas.supports_blit:
                self._background = canvas.copy_from_bbox(self.figure.bbox)

    def _on_draw(self, *args):
        if self.figure:
            canvas = self.figure.canvas
            canvas.mpl_disconnect(self.draw_event_cid)
            if canvas.supports_blit:
                self._set_background()
                self._draw_animated()
            else:
                canvas.draw_idle()
            self.draw_event_cid = canvas.mpl_connect('draw_event', self._on_draw)

    def _draw_animated(self):
        if self.ax.figure and self.figure.axes:
            canvas = self.ax.figure.canvas
            if canvas.supports_blit:
                canvas.restore_region(self._background)
            for ax in self.figure.axes:
                artists = []
                artists.extend(ax.images)
                artists.extend(ax.collections)
                artists.extend(ax.patches)
                artists.extend(ax.lines)
                artists.extend(ax.texts)
                artists.extend(ax.artists)
                artists.append(ax.get_yaxis())
                artists.append(ax.get_xaxis())
                for a in artists:
                    if a.get_animated():
                        ax.draw_artist(a)
            if canvas.supports_blit:
                canvas.blit(self.figure.bbox)

class ImagePlot(BlittedFigure):

    def __init__(self):
        super(ImagePlot, self).__init__()
        self.colorbar = True


    def create_figure(self):
        self.figure = plt.figure()
        self.draw_event_cid = self.figure.canvas.mpl_connect('draw_event',
                                                             self._on_draw)

    def create_axis(self):
        self.ax = self.figure.add_subplot(111)
        self.update_data_cid = self.figure.canvas.mpl_connect('key_press_event',
                                                              self.update)

    def plot(self, **kwargs):
        self.create_figure()
        self.create_axis()
        self.ax.get_xaxis().set_animated(self.figure.canvas.supports_blit)
        self.ax.get_yaxis().set_animated(self.figure.canvas.supports_blit)
        self.figure.canvas.mpl_connect('key_event', self.update)

        self.update(**kwargs)

        if self.colorbar is True:
            self._colorbar = plt.colorbar(self.ax.images[0], ax=self.ax)
            self._colorbar.set_label(
                'quantity_label', rotation=-90, va='bottom')
            self._colorbar.ax.yaxis.set_animated(
                self.figure.canvas.supports_blit)

        self._set_background()
        # ask the canvas to re-draw itself the next time it
        # has a chance.
        # For most of the GUI backends this adds an event to the queue
        # of the GUI frameworks event loop.
        self.figure.canvas.draw_idle()
        try:
            # make sure that the GUI framework has a chance to run its event loop
            # and clear any GUI events.  This needs to be in a try/except block
            # because the default implementation of this method is to raise
            # NotImplementedError
            self.figure.canvas.flush_events()
        except NotImplementedError:
            pass

    def update(self, *args):
        ims = self.ax.images
        # update new data
        data = self.data_function()
        print('update data')
        
        if ims:
            ims[0].set_data(data)
            if self.figure.canvas.supports_blit:
                self._draw_animated()
                # It seems that nans they're simply not drawn, so simply replacing
                # the data does not update the value of the nan pixels to the
                # background color. We redraw everything as a workaround.
                if np.isnan(data).any():
                    self.figure.canvas.draw_idle()
            else:
                self.figure.canvas.draw_idle()
        else:
            self.ax.imshow(data, animated=self.figure.canvas.supports_blit)
            self.figure.canvas.draw_idle()
            
class Signal(object):
    
    def __init__(self):
        pass

    def data_function(self):
        return np.random.rand(4,4)
        
    def plot(self):
        self._plot = ImagePlot()
        self._plot.data_function = self.data_function
        self._plot.plot()
        
a = Signal()
a.plot()

This is a minimal example to reproduce the issue using hyperspy in ipython.

%matplotlib qt
import hyperspy.api as hs
import numpy as np

a = hs.signals.Signal1D(np.random.rand(10*10*100).reshape((10, 10, 100)))
a.plot()

Actual outcome

The animated are missing from the figure in the first draw. Once a draw_event is triggered, everything is plotted on the figure as expected. When resizing the figure, the animated artists are not plotted, triggering a draw_event plots the them. This is working fine with matplotlib 2.1.2, including resizing the figure.

Expected outcome

Plot everything, including the animated artists

Matplotlib version

• Operating system: windows 7, linux
• Matplotlib version: 2.2.2
• Matplotlib backend (print(matplotlib.get_backend())): Qt5Agg
• Python version: 3.6
• Jupyter version (if applicable): 1.0.0
• Other libraries: hyperspy dev (on hyperspy 1.3, there is recursive repaint issue, which is fix in the dev version)

[tacaswell]

With

class BlittedFigure(object):

    def __init__(self):
        self._background = None

    def _set_background(self):
        if self.figure:
            canvas = self.figure.canvas
            if canvas.supports_blit:
                self._background = canvas.copy_from_bbox(self.figure.bbox)

    def _on_draw(self, *args):
        if self.figure:
            canvas = self.figure.canvas
            if canvas.supports_blit:
                self._set_background()
                self._draw_animated(do_blit=False)

    def _draw_animated(self, do_blit=True):
        if self.ax.figure and self.figure.axes:
            canvas = self.ax.figure.canvas
            if do_blit and canvas.supports_blit:
                canvas.restore_region(self._background)
            for ax in self.figure.axes:
                artists = []
                artists.extend(ax.images)
                artists.extend(ax.collections)
                artists.extend(ax.patches)
                artists.extend(ax.lines)
                artists.extend(ax.texts)
                artists.extend(ax.artists)
                artists.append(ax.get_yaxis())
                artists.append(ax.get_xaxis())
                for a in artists:
                    if a.get_animated():
                        ax.draw_artist(a)
            if do_blit and canvas.supports_blit:
                canvas.blit(self.figure.bbox)

it behaves as expected on close-to-master. See warning https://matplotlib.org/api/backend_bases_api.html?highlight=draw_event#matplotlib.backend_bases.DrawEvent (which is probably not the best place for that warning) and discussion at #9406

I'm a bit confused by what version your using ,and which versions it works on.

[ericpre]

Thanks @tacaswell for the quick reply! It is useful and your fix seems work nicely even if I don't think that I understand the reason...

Sorry for the confusion between the different versions, indeed I made a mistake in the title of the issue, which is corrected now. I also edited the first post to clarify the issue. I have been pulling my hair out for too many hours!!

And thanks for pointing out to the relevant issue/documentation on this, we fixed this issue with disconnecting and re-connecting the 'draw_event' at the beginning and the end of _on_draw. Not sure if this is the best but it does the job.

Out of curiosity: why was it working before and it is broken with the last release?

[tacaswell]

There were some tweaks all the way at the bottom of the Qt draw stack that changed exactly how the Qt level that shifted details of when the call to the base draw happened and unified the normal and blitting code paths. Before Agg would render, the draw_event would fire, and then the paint event would happen, now the paint event starts, if it has to re-render Agg renders, draw_event fires (with-in the paint event), and then the paint event exits. Moving the draw_event inside of the paint is what causes trouble for the blitting as it then recurses at the Qt level!

I strongly suggest not doing things inside of _on_draw that can cause another draw.

[ericpre]

What triggers a 'draw_event'? It seems that draw_artist doesn't?

[tacaswell]

It is triggered in Figure.draw

matplotlib/lib/matplotlib/figure.py

Line 1480 in 02646cb

self.canvas.draw_event(renderer)

That method is called on your behalf via the draw method of your canvas ex for Agg based backends:

matplotlib/lib/matplotlib/backends/backend_agg.py

Lines 414 to 433 in 02646cb

	def draw(self):
	"""
	Draw the figure using the renderer
	"""
	self.renderer = self.get_renderer(cleared=True)
	# acquire a lock on the shared font cache
	RendererAgg.lock.acquire()
	toolbar = self.toolbar
	try:
	# if toolbar:
	# toolbar.set_cursor(cursors.WAIT)
	self.figure.draw(self.renderer)
	# A GUI class may be need to update a window using this draw, so
	# don't forget to call the superclass.
	super().draw()
	finally:
	# if toolbar:
	# toolbar.set_cursor(toolbar._lastCursor)
	RendererAgg.lock.release()

It is best to think of the 'draw_event' subscription as "I have just redrawn the figure from scratch at the current DPI and size, but with out anything labeled 'animated' and have not yet put it on the screen"

[ericpre]

Closing since the issue has been fixed in hyperspy/hyperspy#1890. Thanks @tacaswell for the explanation.