Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure qt zmq streams starts in a clean state #307

Merged
merged 3 commits into from Feb 14, 2018

Conversation

minrk
Copy link
Member

@minrk minrk commented Feb 10, 2018

adding a listener on stream.FD won’t reliably wake the eventloop if there are already events waiting to be processed. We must check socket.EVENTS and process any waiting events before waiting on zmq.FD, or we will never wake.

This fixes the bug reported in zeromq/pyzmq#1140

It's caused by updating pyzmq to version 17,
where the tornado eventloop integration also relies on zmq.FD rather than zmq_poll, which allows the initial state of the FD to be consumed (by tornado) while there are events waiting to be processed (because tornado is paused indefinitely).

I think there's something fragile in our multi-eventloop integration because we aren't explicitly advancing the tornado eventloop at all when a GUI eventloop is running, only our kernel.do_one_iteration callback. This means that any callbacks registered directly with tornado/asyncio will not run while eventloop integration is enabled.

cc @ccordoba12 who reported the issue. Can you confirm this fixes it for you?

adding a listener on stream.FD won’t reliably wake the eventloop if
there are already events waiting to be processed.
We must check socket.EVENTS and process any waiting events before zmq.FD will be written-to again.
@ccordoba12
Copy link
Member

Thanks for the quick response Min! I confirm this PR solves the problem I reported in pyzmq.

@@ -41,6 +41,11 @@ def process_stream_events():
fd = stream.getsockopt(zmq.FD)
notifier = QtCore.QSocketNotifier(fd, QtCore.QSocketNotifier.Read, kernel.app)
notifier.activated.connect(process_stream_events)
# there may already be events waiting,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@minrk Double checking my understanding and perhaps tweaking the comment.

there may already be unprocessed events waiting.
these events will not wake zmq's edge-triggered FD
since edge-triggered notification only occurs on new i/o activity.
process all the waiting events immediately 
so we start in a clean state ensuring that any new i/o events will notify.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! updated

with a zero-timeout single-shot timer (closest I could get to .call_soon())
rather than calling it immediately,
which might block in unexpected ways by processing events while we are still hooking things up.
@minrk
Copy link
Member Author

minrk commented Feb 14, 2018

Updated to schedule the first call on the eventloop via a QTimer, rather than making a potentially long-running blocking call immediately which may be a bit early.

@willingc
Copy link
Member

@minrk Looks good. Tested locally with @ccordoba12's setting in the config file. Tests pass and gui appears. Nicely done.

@willingc willingc merged commit 3472ddb into ipython:master Feb 14, 2018
@minrk minrk deleted the process-first-stream branch February 16, 2018 12:57
@stonebig
Copy link
Contributor

stonebig commented Feb 16, 2018

releasing soon 4.8.2 ?

@ccordoba12
Copy link
Member

@minrk
Copy link
Member Author

minrk commented Feb 19, 2018

Just released 4.8.2 with this patch.

@minrk
Copy link
Member Author

minrk commented Feb 19, 2018

conda-forge pending

@ccordoba12
Copy link
Member

ccordoba12 commented Feb 19, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants