Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plt.pause() with threading is extremely slow for MacOSX backend #5675

Closed
danijar opened this issue Dec 14, 2015 · 13 comments · Fixed by #25553
Closed

plt.pause() with threading is extremely slow for MacOSX backend #5675

danijar opened this issue Dec 14, 2015 · 13 comments · Fixed by #25553
Milestone

Comments

@danijar
Copy link

danijar commented Dec 14, 2015

Here is a minimal example where a worker produces a sequence that's plotted every second by the main thread. Between the draw calls, the main thread pauses for 0.001s to update the GUI and sleeps for 1s. This takes 10 seconds. When pausing for 1s and sleeping for 0.001s instead, I would expect this to run in about the same time. However, it takes 70 seconds.

import random
import time
import threading
import matplotlib.pyplot as plt


class Plot:

    def __init__(self):
        plt.ion()
        plt.show()
        self.xdata = []
        self.ydata = []
        self.running = None
        self.axis = plt.figure().add_subplot(111)
        self.line, = self.axis.plot([])

    def start(self):
        self.running = True
        threading.Thread(target=self.worker).start()
        while self.running:
            self.axis.set_xlim(0, len(self.xdata))
            self.axis.set_ylim(0, max(self.ydata))
            self.line.set_data(self.xdata, self.ydata)
            plt.draw()
            plt.pause(0.001)
            time.sleep(1)

    def worker(self):
        for _ in range(100):
            self.xdata.append(len(self.xdata))
            self.ydata.append(random.random())
            time.sleep(0.1)
        self.running = False


if __name__ == '__main__':
    start = time.time()
    Plot().start()
    print(time.time() - start)

Profiling revealed that by far the most time is spent in start_event_loop of _macosx.FigureCanvas.

Possibly related: #5251

@jenshnielsen
Copy link
Member

can you try the fix in #5562?

@mdehoon
Copy link
Contributor

mdehoon commented Dec 15, 2015

There are two issues here:

  1. Drawing should be done in the main thread, not in the worker thread;
  2. You don't need these lines:
            plt.draw()
            plt.pause(0.001)
            time.sleep(1)

The following code works as intended:

import random
import time
import threading
import matplotlib.pyplot as plt


class Plot:

    def __init__(self):
        plt.ion()
        plt.show()
        self.start = time.time()
        self.xdata = []
        self.ydata = []
        self.running = None
        self.axis = plt.figure().add_subplot(111)
        self.line, = self.axis.plot([])
        threading.Thread(target=self.worker).start()

    def worker(self):
        for _ in range(100):
            self.xdata.append(len(self.xdata))
            self.ydata.append(random.random())
            self.axis.set_xlim(0, len(self.xdata))
            self.axis.set_ylim(0, max(self.ydata))
            self.line.set_data(self.xdata, self.ydata)
            time.sleep(0.1)
        print(time.time() - self.start)


if __name__ == '__main__':
    Plot()

@mdehoon
Copy link
Contributor

mdehoon commented Dec 15, 2015

To explain how this works:
The event loop starts running in the main thread as soon as the MacOSX backend is imported.
Then we start the following cycle:

  • The event loop is sitting in the main thread, waiting for something to happen;
  • The worker thread calls set_xlim, set_ylim, set_data, which will invalidate the canvas;
  • The event loop notices that the canvas is invalidated, triggering a redraw in the main thread.

@koogle
Copy link

koogle commented Dec 15, 2015

Thanks for the explanation. I tried the updated script but the window does not get redrawn at all.
Which version of matplotlib are you referring to? (I am using 1.5.0)

@mdehoon
Copy link
Contributor

mdehoon commented Dec 16, 2015

I am using matplotlib 1.5.0. Run with "python -i script.py"; "python script.py" won't work.

@koogle
Copy link

koogle commented Dec 16, 2015

Okay, I can confirm it works without problems in interactive mode. Thanks for the clarification.

@mdehoon
Copy link
Contributor

mdehoon commented Dec 16, 2015

The reason it doesn't work in non-interactive mode is probably that the set_xlim, set_ylim, set_data doesn't invalidate the canvas in non-interactive mode.

@tacaswell Perhaps stale should always propagate irrespective of whether matplotlib is interactive or non-interactive.

@danijar
Copy link
Author

danijar commented Dec 19, 2015

Thanks @mdehoon for the example @koogle for testing. What would be the right way to make your example work in non-interactive mode?

@mdehoon
Copy link
Contributor

mdehoon commented Dec 21, 2015

@danijar See my comment to @tacaswell above. That should do it.

@danijar
Copy link
Author

danijar commented Dec 21, 2015

As I understand it, your comment suggested a change in matplotlib. Do you have a fix for users that works right now? I thought about invalidating the canvas with something like fig.canvas.draw_idle() but I can't test on Mac right now.

@tacaswell
Copy link
Member

Stale does propagate, the question is if anything at the top is listening.

For non-interactive backends (ex Agg) the draw_idle is not (and can not be) asynchronous (it is re-entrant so stale flags that arise in a draw do not trigger recursive draws).

One possible solution to this is to make the OSX backend's Figure always invalidate it's canvas independent of the interactive state.

Having said that, that might be the right place to put all of this logic (backend dependent).

@github-actions
Copy link

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

@github-actions github-actions bot added the status: inactive Marked by the “Stale” Github Action label Mar 20, 2023
@greglucas
Copy link
Contributor

It seems like there is still some work here to do. This only seems to work as expected on the qt5agg backend for me.

tk: runtime error with calling draw_idle() from off the main thread. (Stemming from setting the figure stale during the xlim updates)

macosx: no canvas updates, it seems like the loop isn't receiving and processing the draw_idle() timers. The canvas is somewhat disassociated with the events. For example, after the worker thread is done, the mouse hovers all work as expected, but the draws never get triggered and update. However, if you click the subplot tool and mess around with those settings, then the Agg canvas is updated and redrawn, but going back to the pan/zoom tools does nothing again, so those events don't seem to go back to the Agg redraws.

@github-actions github-actions bot removed the status: inactive Marked by the “Stale” Github Action label Mar 22, 2023
@QuLogic QuLogic added this to the v3.8.0 milestone Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants