ipython 5.1 sporadically freezes when matplotlib plots are active #10201

Closed
keflavich opened this Issue Jan 25, 2017 · 9 comments

Comments

Projects
None yet
4 participants
@keflavich

I've using ipython 5.1 with matplotlib 1.5.1 with the Qt4Agg backend on Mac OS X 10.9.5. There have now been 3 occasions when the ipython terminal froze entirely - it would not accept control-c, control-z, or any command I tried. Quitting the Qt plot windows re-activated the python terminal.

This so far has not affected me the first or second time I've made a plot window, but eventually it seems to affect every ipython session I've started with a plot window.

I upgraded to matplotlib 2.0, and the same thing happened, but this time (now with Qt5Agg as a backend), I get a message:

QEventDispatcherMac::unregisterSocketNotifier: Tried to unregister a not registered notifier

After that notification, the entire terminal containing ipython became unresponsive. I think the same message was issued under ipython + matplotlib 1.5.1, but the message only appeared after closing ipython entirely, i.e.:

In [11]:
Do you really want to exit ([y]/n)?
QEventDispatcherMac::unregisterSocketNotifier: Tried to unregister a not registered notifier
@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Jan 26, 2017

Member

Ping @tacaswell

Have you run a %matplotlib command to integrate the event loops? What backend are you using?

Member

takluyver commented Jan 26, 2017

Ping @tacaswell

Have you run a %matplotlib command to integrate the event loops? What backend are you using?

@keflavich

This comment has been minimized.

Show comment
Hide comment
@keflavich

keflavich Jan 26, 2017

No, I did not run %matplotlib in any of these sessions, but I started up ipython with --matplotlib.

The two different cases described above are with Qt4Agg and Qt5Agg backends.

No, I did not run %matplotlib in any of these sessions, but I started up ipython with --matplotlib.

The two different cases described above are with Qt4Agg and Qt5Agg backends.

@tacaswell

This comment has been minimized.

Show comment
Hide comment
@tacaswell

tacaswell Feb 5, 2017

Contributor

My naive guess is that something is wrong with the way the eventloops are being hooked together (but only on mac).

Contributor

tacaswell commented Feb 5, 2017

My naive guess is that something is wrong with the way the eventloops are being hooked together (but only on mac).

@keflavich

This comment has been minimized.

Show comment
Hide comment
@keflavich

keflavich Feb 13, 2017

Is there anything I can do to test that hypothesis or work around it?

Is there anything I can do to test that hypothesis or work around it?

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Feb 13, 2017

Member

The code handling Qt event loop integration in terminal IPython lives here:
https://github.com/ipython/ipython/blob/master/IPython/terminal/pt_inputhooks/qt.py

I don't understand what might be going wrong, though.

Member

takluyver commented Feb 13, 2017

The code handling Qt event loop integration in terminal IPython lives here:
https://github.com/ipython/ipython/blob/master/IPython/terminal/pt_inputhooks/qt.py

I don't understand what might be going wrong, though.

@tacaswell

This comment has been minimized.

Show comment
Hide comment
@tacaswell

tacaswell Feb 13, 2017

Contributor

Where does the context object come from?

I wonder if there is a race condition between triggering the inputhook, the socket being ready to be read, the notifier being created, and the loop starting?

The story here would be things fail when:

  • inputhook gets called
  • the context file descriptor is ready to be read and then torn down
  • the notifier in created, but is broken due to the previous step (this is the dicey part)
  • the Qt event loop now starts, but the notifier that needs to stop it to let the terminal work again is broken so the loop Qt event loop runs forever.
  • closing all of the plot windows (which should still be responsive to pan/zoom and keyboard inputs?) implicitly stops the event loop, returning control to the terminal

From work with asyncio I have the general impression that select on OSX is wonky (which I assume something related to that is being under the hood here).

I suspect that the fix will be to notice when constructing the notifier fails and not starting the event loop. It might also make sense to also hook up a timer that kills the event loop after ~0.1s?

Contributor

tacaswell commented Feb 13, 2017

Where does the context object come from?

I wonder if there is a race condition between triggering the inputhook, the socket being ready to be read, the notifier being created, and the loop starting?

The story here would be things fail when:

  • inputhook gets called
  • the context file descriptor is ready to be read and then torn down
  • the notifier in created, but is broken due to the previous step (this is the dicey part)
  • the Qt event loop now starts, but the notifier that needs to stop it to let the terminal work again is broken so the loop Qt event loop runs forever.
  • closing all of the plot windows (which should still be responsive to pan/zoom and keyboard inputs?) implicitly stops the event loop, returning control to the terminal

From work with asyncio I have the general impression that select on OSX is wonky (which I assume something related to that is being under the hood here).

I suspect that the fix will be to notice when constructing the notifier fails and not starting the event loop. It might also make sense to also hook up a timer that kills the event loop after ~0.1s?

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Feb 13, 2017

Member

The context object comes from prompt_toolkit.

Member

takluyver commented Feb 13, 2017

The context object comes from prompt_toolkit.

@tacaswell

This comment has been minimized.

Show comment
Hide comment
@tacaswell

tacaswell Feb 17, 2017

Contributor

who is the right person from prompt_toolkit to loop into this?

Contributor

tacaswell commented Feb 17, 2017

who is the right person from prompt_toolkit to loop into this?

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Feb 17, 2017

Member

@jonathanslenders is the author of prompt_toolkit. I'd suspect the error is more likely to be in our qt inputhook code, but maybe he can help us to work it out.

Member

takluyver commented Feb 17, 2017

@jonathanslenders is the author of prompt_toolkit. I'd suspect the error is more likely to be in our qt inputhook code, but maybe he can help us to work it out.

tacaswell added a commit to tacaswell/ipython that referenced this issue Feb 18, 2017

FIX: re-order qt eventloop hook a bit
try to reduce race conditions by:

 - connect the exit callback before enabling the notifier (this might
   not matter)
 - only bother starting the event loop if the input is not ready.

closes #10201

@Carreau Carreau added this to the 5.3 milestone Feb 21, 2017

@Carreau Carreau closed this in #10301 Feb 21, 2017

@tcrensink tcrensink referenced this issue in matplotlib/matplotlib May 2, 2018

Closed

matplotlib hangs on plotting #11152

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment