-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Accept explicit contextvars.Context in asyncio create_task() API #91150
Comments
Now asyncio creates a new context copy on task creation. It is the perfect behavior *by default* and should stay as is. However, sometimes passing an explicit context into a task and using back the context modified by task code is desired. The most obvious use-case is testing: both unittest and pytest have multi-phase test initialization (asyncSetUp() methods and async fixtures correspondingly). If asyncSetUp() updates context variable -- test code expects to see this change. Currently, unittest runs the setup-test-cleanup chain in a single task and uses an internal queue to push a new coroutine for execution and get results back. It works but is cumbersome. Another solution is to create a task per test execution step and wrap the task creation with Context.run(). The problem is in getting the updated context back. A task creates a context copy on starting, thus the modified context is stored in the task internal attribute only. To get it back a trampoline async function should be used, e.g. async def wrapper(coro):
try:
return await coro
finally:
context = contextvars.copy_context()
# store the context copy somewhere Again, it looks more complicated as it should be. The proposal is:
The proposal is backward-compatible. Low-level API (call_soon(), call_later() etc.) already accept 'context' argument. The attached PR demonstrates how the proposed API simplifies unittest.IsolatedAsyncioTestCase internals. |
Yeah, +1 to add a parameter. Fwiw it was on my idea list when i was working on the pep |
Nice enhancement! |
Any idea how this could be backported to older Python versions? I'm using 3.8. The best I've come up with is lock = threading.Lock()
def create_task_with_context(coro, *, name= None, context = None):
if context is None:
return asyncio.create_task(coro, name=name)
this_thread = threading.get_ident()
def patched():
if threading.get_ident() == this_thread:
return context
else:
return original_copy_context()
with lock:
original_copy_context = contextvars.copy_context
contextvars.copy_context = patched
try:
task = asyncio.tasks._PyTask(coro, name=name)
finally:
contextvars.copy_context = original_copy_context
return task Which is a horrible hack. Any better ideas? Edit: Added thread safety check. |
Never mind the hackiness, this is outright dangerous. What if there's another event loop running in another thread, doing the same exact thing? |
|
The GIL can switch threads any time between byte code instructions. Here's the disassembled byte code for that function of yours:
|
Depends on the definition of backwards compatible, I guess? This particular change broke AnyIO's tests with uvloop: agronholm/anyio#439 (not that I'm against this addition, but...). |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: