[query] ensure nest_asyncio is applied, but only when necessary #13899
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CHANGELOG: Fix
RuntimeError: This event loop is already running
error when running hail in a Jupyter Notebook.Man this is really complicated.
OK, so, things I learned:
set_event_loop
has been called even ifset_event_loop(None)
has since been called.aiohttp.ClientSession
stashes a copy of the event loop present when it starts. This can cause all manner of extremely confusing behavior if you later change the event loop or use that session from a different thread.The fix, in the end, wasn't that complicated. Anywhere Hail explicitly asks for an event loop (so that we can run async code), we apply nest asyncio if the event loop is already running. Otherwise we do nothing. Nest asyncio appears to no longer require
apply
to be called before the event loop starts running.This PR does not address:
hailtop.fs
andhailtop.batch
APIs, among others, need async versions before we can do that.aiohttp.ClientSession
nonsense. We really should take pains to ensure we create oneClientSession
per loop and we never mix loops.