Kernel execution / event loop interactions

[DinoV]

Currently ipykernel has a Tornado event loop that it used to process incoming requests. This event loop is running on the main thread of the process - ultimately that seems to be a requirement as Python only allows raising a KeyboardInterrupt on the main thread (which gets accomplished by either sending SIGINT to the kernel process or by calling interrupt_main on Windows).

ipykernel also spins up a 2nd thread, with a 2nd event loop, for iopub events which "Prevents a blocking main thread from delaying output from threads."

Unfortunately this doesn't play nicely with moving the debug protocol to run over Jupyter channels. When an execution occurs the main event loop is actually blocked on the execution. If we were then to signal that we hit a breakpoint, tried to pause execution, etc... the shell IO event loop isn't running to process debugger commands.

Therefore it seems like the shell event loop needs to move to it's own thread as well, and then have the executions continue to proceed on the main thread. The event loop will need to schedule executions on the main thread.

This could also tie back to the issues raised in issue #8. If we went with option 1 in there then the fact that we block the IO pub event loop isn't such a big deal as the debugger now has it's own event loop, similar to IOPub. It could also be the case that other kernel implementations are going to be similarly impacted given that there seems to be an expectation that their shell message processing can become blocked while executing.

[Carreau]

I have a long-term plan to completely update IPython itself, and likely ipykernel to be sans-io and agnostic as to whether it's running in trio/asyncio/curio/uvloop. We have other threading problem, (like with completion that regularly does not like to be ran in separate thread).

If that needs to be prioritized for debugger purpose, we can do that, though there is a ton of things that are blocking right now and would be a massive amount of work.

[DinoV]

Would that involve moving the event loops onto their own thread and keeping the execution on the main thread?

If that's part of the solution it might be that I can look into doing at least that part but if you had something else in mind I can defer to those grander plans.

[Carreau]

Well in the best case scenario, if we can manage to get everything sans-io then in many case there would not be a need for separate threads for the core to run (AFAICT, but i'm not that well verse in ZMQ). The biggest blocking point so far is that Python does not have async-exec so we would have to fake it, and if the user code is blocking then the main thread may still be blocking in which case we would still need a second thread.

But I expect most of the current limitation to be consequences of the fact that input() blocks in terminal IPython, and that once this is taken care of many design decisions could be simplified.

[DinoV]

The issue with the debugger is that it does need to block... Consider something simple like:

while True:
    print('hi')

If a user hits the pause button, then we're blocked on one of those two lines, just as if we were blocked in input(). The only way around blocking would be if you could unroll the stack and support restoring it, but something like that would require stackless.

This can certainly move to an event driven system as part of moving to sans-io, with the break-in being delivered as an event to some event sink, but then the debugged thread is going to need to block (or worse poll) for the resume event to come back. And that resume event will need to come back on another thread.

I suspect input() will fall under the exact same sort of situation... Calling input() will raise an event for receiving input, and the call to input() will necessarily need to block to prevent forward progress, and then the event will need to come in from another thread to unblock it and return the input.

I suspect for sans-io the kernel needs to expose an event based system, but those events need to be thread safe and can be raised or delivered on multiple threads. In the case of something like tornado which is it's self designed to be single threaded that might mean some additional thread safety (e.g. everything needs to dispatch onto it's event loop via loop.add_callback) and other event loops will need to deal w/ their own threading as appropriate.

Does that sound right?

[SylvainCorlay]

@DinoV you might be interested in how we deal with concurrency in the Xeus implementation of the kernel protocol. Xeus is a C++ implementation of the protocol that is meant to simplify the authoring of kernels.

To create a new kernel based on Xeus, all you need is to specialize the base xinterpreter virtual class as shown in the documentation. It would be fairly easy to make a xeus-based Python kernel embedding the Python interpreter and making use of it as a library. We used Xeus as the foundation for the C++ kernel that we announced earlier in this post.

The reason why it may be interesting for you is that Xeus's concurrency model is a bit different from ipykernel.

• It uses operating systems' threads. The default implementation is that there are two threads running the heartbeat and the iopub message handling, while the main thread handles the control and shell channels.
• You can easily change the usage scenario by specializing the xserver base class even though we provide a default implementation for the scenario described above with native threads for iopub and heartbeat channels.

The xserver is responsible for providing the middleware logic, and a specialized implementation could follow a different logic from the one described above without changing the implementation of the interpreter... This could be a good way for you to experiment with other concurrency models for debugging without changing the language-specific part of the implementation (xinterpreter).

cc @JohanMabille @ellisonbg @Carreau @minrk