Updating a plot doesn't clear old plots, if event trigger came from another figure?

[clsmt]

Bug report

Bug summary

I am updating a line plot. There is an event trigger that starts this updating. If the trigger came from the figure that contains the plot, everything is fine. However, if the trigger came from another figure, then weird results happen: the line that's been updated appears to leave its trace uncleared.

Code for reproduction

import matplotlib.pyplot as plt
import numpy as np

def onclick(event):

    for ii in np.linspace(0., np.pi, 100):
        y1 = y * np.sin(ii)
        line1.set_ydata(y1)
        ax.draw_artist(line1)
        line2.set_ydata(-y1)
        ax2.draw_artist(line2)
        ax2.set_ylim(y1.min(), y1.max())
        fig.canvas.update()
        plt.pause(0.1)


x = np.linspace(0., 2*np.pi, 100)
y = np.sin(x)

fig = plt.figure()
ax = fig.add_subplot(1, 2, 1)
line1 = ax.plot(x, y)[0]

ax2 = fig.add_subplot(1, 2, 2)
line2 = ax2.plot(x, y)[0]


fig2 = plt.figure()

cid = fig2.canvas.mpl_connect('button_press_event', onclick)
# cid = fig.canvas.mpl_connect('button_press_event', onclick)

plt.show()

Actual outcome

Expected outcome

Not the many curves saw in the above figure, just one curve being updated in each cycle. In the code snippet, if use

cid = fig.canvas.mpl_connect('button_press_event', onclick)

then it works as intended.

Also note, if you resize the plot, or save it as figure, then all the residue curves will be gone.

Matplotlib version

• Operating system: windows
• Matplotlib version: 2.0.0
• Matplotlib backend (print(matplotlib.get_backend())): Qt4Agg
• Python version: 2.7
• Jupyter version (if applicable):
• Other libraries:

[anntzer]

You should use fig.canvas.draw() (a matplotlib method), not update() (a Qt method). Because the call was to update(), the window was not cleared first... except that pause() itself triggers a draw(), but only on the active window (and is explicitly documented as such), which is the one you clicked on:

def pause(interval):
    """
    Pause for *interval* seconds.

    If there is an active figure, it will be updated and displayed before the
    pause, and the GUI event loop (if any) will run during the pause.

    This can be used for crude animation.  For more complex animation, see
    :mod:`matplotlib.animation`.

    Notes
    -----
    This function is experimental; its behavior may be changed or extended in a
    future release.
    """
    manager = _pylab_helpers.Gcf.get_active()
    if manager is not None:
        canvas = manager.canvas
        if canvas.figure.stale:
            canvas.draw_idle()
        show(block=False)
        canvas.start_event_loop(interval)
    else:
        time.sleep(interval)

I guess pause() could be changed to redraw all pyplot-managed windows (so leaving this issue open for discussion).

[clsmt]

For my actual application, draw() is way too slow, and I can't use it. I am kinda stuck with update() cause it's much faster. Any suggestions?

[tacaswell]

@clsmt You have half re-invented blitting, see https://matplotlib.org/api/animation_api.html?highlight=blitting#funcanimation

The thing you are missing is caching and restoring the figure without your animated artist on it.

[tacaswell]

and if you want to change the limits you will have to re-draw the whole axes (all of the artists in it + the tick labels).

[clsmt]

@tacaswell
I am not sure if I understand it correctly -

1. Does update() do blitting in the backstage for me?
2. If I redraw the whole axes, the performance is going to be slow, right? But it seems update() is still much faster with redrawing limits etc. Why?

[tacaswell]

No, update is a Qt object method that schedules a paint event for the near future which eventually results in Qt calling the paintEvent method which copies pixels values from the Agg buffer to the screen (the Qt backend is implemented as QImage that is painted onto the widget).

When you call draw_artist that renders the artist onto the current Agg buffer (without clearing anything currently on the canvas). The subsequent call to update eventually copies the updated buffer to the screen. If you use blit it uses repaint under the hood which updates the screen now, rather than when the Qt paint event make's it's way to the top of the GUI event loop.

In the right-hand axes I am not quite sure what is going on to end up with those ticks, but each curve is drawn with a different set of limits (none of which match the tick labels).

Below is code that does what I think you want.

import matplotlib.pyplot as plt
import numpy as np
import time
bg_cache = None
current_timer = None


def onclick(event):
    global current_timer
    j = 0
    cv = event.canvas
    fig = cv.figure
    frames = 100

    def update():
        nonlocal j
        # restore the background
        cv.restore_region(bg_cache)

        # update the data

        ii = j * np.pi / frames
        y1 = y * np.sin(ii)

        # update the line on the fixed limit axes
        line1.set_ydata(y1)
        ax.draw_artist(line1)

        # update the line on the limit adjusted axes
        line2.set_ydata(-y1)
        ax2.set_ylim(y1.min(), y1.max())
        fig.draw_artist(ax2)

        # update the screen
        cv.blit(fig.bbox)
        # update counter
        j += 1
        # if we are more than 100 times through, bail!
        if j > frames:
            print(f'fps : {frames / (time.time() - start_time)}')
            return False

    start_time = time.time()
    current_timer = cv.new_timer(
        interval=100, callbacks=[(update, (), {})])

    current_timer.start()


def ondraw(event):
    global bg_cache
    cv = event.canvas
    fig = cv.figure
    bg_cache = cv.copy_from_bbox(fig.bbox)

    ax.draw_artist(line1)
    fig.draw_artist(ax2)


x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)

fig = plt.figure()
ax = fig.add_subplot(1, 2, 1)
line1, = ax.plot(x, y, animated=True)

ax2 = fig.add_subplot(1, 2, 2)
ax2.set_animated(True)
line2, = ax2.plot(x, y)


d_cid = fig.canvas.mpl_connect('draw_event', ondraw)
bp_cid = fig.canvas.mpl_connect('button_press_event', onclick)

plt.show()

The various globals are required so things do not get garbage collected. The two callbacks should probably be bound into a class and some checking like (if a timer is running, don't start another one).

For a 640x480 pix figure I can get ~60 fps and 200fps if only updating the left axes (if you turn the interval down to 1ms) on a 4 year old laptop.

[tacaswell]

@clsmt Do you find the text at #10187 helpful?

[clsmt]

@tacaswell Thanks a lot for the reply. It is very helpful to me. I think I had several misunderstandings before this thread, and thanks to all the help I got here I could clear them up. I will write those as notes (with some more questions) here in case someone else had the same confusions.

1. plt.pause() actually calls draw_idle(). I saw some random online snippets that do canvas.draw() then plt.pause(), which would actually be duplicate.
2. I think in many cases why you need plt.pause() is because you need flush_events(). You can call it directly. Speaking of which, I am not sure I entirely understand flush_events(), I can find limited information about this online. the document says 'Flush the GUI events for the figure.' Does this just give the backend a chance to do the plotting?
3. There is no actual 'updating' in plotting. What I mean is this: new things get plotted to the canvas, with all the stuff plotted before still on it. Once an artist gets plotted to the canvas, the handle to the artist has no bearing on the curve that's plotted on the canvas. I had this misunderstanding for a long time; you can see in my original snippet it could be interpreted wrongly that things are 'updated' in the sense of 'moved'. No, draw_artist(artist1) does a new plotting of artist1 to canvas (by renders the artist onto the current Agg buffer). canvas.update() which is a Qt method schedules this plotting event (in the event loop?)
4. @tacaswell 's reply is a great snippet to teach blitting. IMHO it's actually better than the official document found at https://matplotlib.org/api/animation_api.html. The official document is rather abstract, and it's associated with FuncAnimation which I don't use. I have another question about blitting: from the official doc it seems the reused background is a bitmap? It's not actually pixelated bitmap that's being reused, right? (In the sense that you can still save the figure as the vector format.)
5. So back to my original question, there are two easy solutions: one is blitting as in @tacaswell's answer. Another option is do the draw_artist(ax.patch) just before updating the line. This re-draw the empty space in the axes, and cover the original plots.

I am an end user of matplotlib, and I don't have much knowledge of the backstage stuff. I learned through scraping information online and reading bits and pieces of the documentation. If anyone out there is in a similar situation and end up with similar confusions, I hope this thread helps you.

[clsmt]

Ok, one more questoin: why is it that in my original code, after all the updating is done, when you resize the firgure, all the previous curves but the latest are gone?

[tacaswell]

1. yes, but as of 1.5 draw_idle only actually re-draws if the figure is 'stale'
2. see DOC: Start to document interactive figures #4779 for lots of details, but in short the actual render is delay until the GUI framework decides to update which in most cases is via an event on the GUI event loop which requires the event loop to run.
3. Matplotlib is split in to layers, one layer (the Artists holds the logical representation of the plot (the Line2D objects) and the renders (which take the Artists and turn them into some manifested version of the plot). Agg is a compositing renderer so it's bit map is just the accumulation of all the artists that have been drawn to it (but with no further connection) so calling draw_artist just changes pixels in the Agg buffer and (as you say) update (in a round about way) moves that buffer to the screen. See http://www.aosabook.org/en/matplotlib.html
4. I moved a version of that snippet to DOC: add a blitting tutorial #10187 , is it still helpful? Yes, with blitting you are storing a bit-map of the background and then using that to 'clean' the Agg buffer (which is also a bit map). If you want to save the figure as a vector format, the figure is rendered from scratch from the Artists to the vector format based on their current state. This is the same reason resizing the figure clears the extra lines. When the figure is re-sized the number of pixels changes so it is automatically re-rendered from scratch based on the current state of the artists, hence giving you only one line
5. yes, but blitting is more robust against things like having non-animated artists in the axes (like annotations, inward facing ticks, other lines, etc) and partially transparent patches.

Where would you have expected to look in the documentation to have found this information? Do the two PRs linked above make it better or worse?

[tacaswell]

I would also consider using FuncAnimation it does all of the blit management behind the scenes for you and by separating the update-my-artists logic from the source-my-data logic will (hopefully) lead to slightly easier to maintain and re-use code for you.

[clsmt]

Deepest apologies for not responding sooner.

1. I've tried to use FuncAnimation several times, but with left it in the end. The reason being it's somewhat intrusive in the thinking process of designing my package. I have to work around FuncAnimation, not just "now plot A and B" then "let's update B". In my code, the user would be doing computations and then updating the plots accordingly (but I don't know when). Not just playing a certain animation and be done. Maybe it's just my way of thinking.

2. Your example above is very good. I wish I had read this before I wrote my last project.

3. I am not sure where should this info go. Your explanation of the layers, Artists vs Agg, is very helpful to me. The explanation of blitting and "updateing" in the document was kind of confusing for me, since I have no prior exposure to these things. I would think explaining the actual plotting flow here would make a lot of sense.

4. A blitting tutorial would be great as well.

[clsmt]

A further problem with set_animated, if the ax and the figure are created in a class, the ax won't show up, it's missing from the figure. Try the following code to reproduce:

import matplotlib.pyplot as plt
import numpy as np

class Plotter:

    def __init__(self):
        self.fig = plt.figure()
        self.ax = self.fig.add_subplot(111)
        self.ax.set_animated(True)

x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)

plotter = Plotter()
plotter.ax.plot(x, y)
plotter.fig.draw_artist(plotter.ax)

plt.show()

[tacaswell]

This is not related to being in the class, you can reproduce it via

import matplotlib.pyplot as plt
import numpy as np


fig = plt.figure()
ax = fig.add_subplot(111)
ax.set_animated(True)

x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)


ax.plot(x, y)
fig.canvas.draw()
fig.draw_artist(ax)
plt.show()

The issue is that draw_artist renders the artists onto the internal buffer, but does not move the rendered buffer to the screen, for that you need to use blit as above:

fig.canvas.blit(fig.bbox)

[clsmt]

fig.canvas.blit(fig.bbox)

where should I place this line? I tried place it after fig.canvas.draw(), then I still get nothing. If placed before that, I get AttributeError: 'FigureCanvasQTAgg' object has no attribute 'renderer'.

Also, in your example above , why do the axes show up initially? Is it because ax is not 'animated', only line1 is?

line1, = ax.plot(x, y, animated=True)

Also in the example, cv.blit(fig.bbox) only draws those that were set to be animated, i.e., line and ax2. Is that correct? I tried setting them not to be animated, i.e., line1, = ax.plot(x, y, animated=False), ax2.set_animated(False). Seems the animation still works, with no fps change.

The Qt method update() does blitting internally. Thus, whether I do blitting explicitly myself, or do it via update(), I shouldn't expect much speed difference, is that correct?

In my original example, why does it matter where the event trigger is from the same figure or not? Is it because of the way update() works internally?

[QuLogic]

This seems to be more like a user question than a feature request or a bug report, so I'm going to close this. You can continue any questions on https://discourse.matplotlib.org/