Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep async event loop alive between cells #11829

Open
mehaase opened this issue Jul 17, 2019 · 1 comment

Comments

@mehaase
Copy link

commented Jul 17, 2019

The %autoawait magic is amazing! It works very smoothly with asyncio and I've had a great time using it.

When it comes to Trio, however, the current integration is pretty limited, because Trio has a notion of structured concurrency, which means that all concurrent tasks must belong to a specific parent (called a "nursery" in Trio parlance). Therefore, if an object needs to run any tasks (e.g. a connection object that needs a background task to read incoming network data), that object's lifetime is bound to a nursery, and a nursery's lifetime is bound to the block of a context manager. The effect is that any object that needs to spawn a long-lived Trio task can only be used within a single cell.

Here's an example:

trio example in jupyter notebooke

If I execute the second cell here, it will hang for a long time. Even though the connect_websocket_url() returns relatively quickly, the async with block will stay open as long as the connection's background task is running, i.e. as long as the connection itself is open.

On the prototype async REPL issue, @njsmith mentioned the following:

But we need to eventually reach a point of keeping the same loop alive over multiple cells, if we want this to be really useful.

I couldn't find any open issues tracking this idea to keep an event loop open between cells, so I'm opening this issue. I'm not sure how this would work in general, but for Trio specifically we probably want to call trio.run() once, run each cell as a task on this single Trio loop, and expose a global nursery object for launching background tasks that keep executing even when a cell isn't currently running.

I can contribute some time to helping out with this. I poked around some of the PRs and source code, but I'm not very familiar with IPython's internals, so I figure the best place to start is to brainstorm with those of you who understand the issues better than me.

@Carreau

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

Thanks for opening this issue, and for the kind words !

There are actually two sides of this issue:

  • in ipykernel tornado is running, so we likely need to swap the eventloop, or integrate both.
  • In terminal IPython we need to make this work.

I personally don't have enough time to goo deep the rabbit hole on these, but will do my best to help anyone with the motivation to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.