-
Notifications
You must be signed in to change notification settings - Fork 10.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent edge case that creates extra event loop #5832
Conversation
`asyncio.get_event_loop_policy().get_event_loop() is asyncio.get_running_loop()` is not always true, as mentioned in python/cpython#96377 (comment). The AsyncioSelectorReactor runs in the second thread and uses the already initialised event loop in the main thread to run the crawler, so a single event loop will be running code from two threads.
I expect tests to break, we had to change this recently for a reason. Let’s see… |
Codecov Report
@@ Coverage Diff @@
## master #5832 +/- ##
==========================================
- Coverage 88.94% 88.84% -0.11%
==========================================
Files 162 162
Lines 11002 11055 +53
Branches 1798 1800 +2
==========================================
+ Hits 9786 9822 +36
- Misses 937 954 +17
Partials 279 279
|
Replace `get_asyncio_event_loop_policy` with `set_asyncio_event_loop_policy` to minimise usage of `asyncio.get_event_loop_policy`. If the issues with the default loop on Windows are resolved, `set_asyncio_event_loop_policy` will be redundant and so will all the policy functions of asyncio.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving, though I wonder if this can break some other rare use case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any chance you can add a test that only passes after your changes? To make sure we do not break what you are fixing here later on.
Thanks! |
Fixes #5831.
asyncio.get_event_loop_policy().get_event_loop() is asyncio.get_running_loop()
is not always true, as mentioned in python/cpython#96377 (comment). The AsyncioSelectorReactor runs in the second thread and uses the already initialised event loop in the main thread to run the crawler, so a single event loop will be running code from two threads.According to python/cpython#100160, we might need to replace
asyncio.get_event_loop()
withasyncio.get_running_loop()
if semantics change. Currently, some low level aspects of asyncio are in flux.