Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
asyncio support #2
Makes sense. I'll wait for the author's signal to do it, if you don't mind. Em sáb, 28 de abr de 2018 06:15, Fantix King <firstname.lastname@example.org> escreveu:…
Could we please keep this issue open before asyncio support lands? I'd like to reference this issue, but a closed one seems a little bit weird.
I've looked into this issue for solution, and found it tricky and requiring tradeoffs perhaps.
Backport python/cpython#5027 as it is, by providing a patched asyncio event loop (as well as
In CPython 3.7 contextvars works across different Tasks in a way that, each step of a Task is executed explicitly in the same Context owned by this Task. Therefore when multiplexing several Tasks, the current running Task always pushes its own Context into the stack first, then run the next step, then pop it out before switching back to event loop for the next Task to run. So that for the coroutines driven by Tasks, they feel like the Context is local to itself and isolated from other coroutines/Tasks. (Please correct me if my understanding was wrong)
To follow that in this backport, a patched Task would be essential to own a Context and drive steps within it. Therefore,
So I thought about what context actually means for asyncio:
This does not cover 100% of PEP-567 for asyncio, but I think it is the most important part. The idea was from PEP-550, we had some discussion in fantix/gino#84, and it led to aiocontextvars. Approach 2 is very much the same:
It has the minimal impact, while providing the key feature. However, the defects are also obvious and fatal:
Therefore I think Approach 2 is definitely a no-go, and listed here just for inspiration.
I'd love to see a different third approach here. Please shed some light on this, and I'm willing to contribute PoC/PR on it.
@fantix , I know I'm just a spectator here, but as I understood from your ideas and some thoughts about:
Approach 1: We would end up with "patched" versions of
try: from contextvars import asyncio, ContextVar # for Python 3.6 and 3.5 except ImportError: import asyncio from contextvars import ContextVar
Is that correct? IMO this works just fine, since I'm already hooking some task factories to manage some kind of context variables (Python 3.5 and 3.6). Even made a simpler approach to use with sanic-jwt on some specific occasions, so developers worried with this backport may already know they need to get their hands a little dirty ...
Approach 2: as you said in the end, a "no-go". But I think that aiocontextvars is pretty much usable (and works with custom event loops, doesn't it?). So, my question would be: why not merge them and provide additional documentation on how to use them? I have a footnote explaining myself a little bit.
Approach 3: Well, I think I already made some thoughts on the first two approaches. Even a mashup of both would be interesting to see - if they provide this functionality, of course!
1 I'm starting to think - @1st1 and @fantix, correct me if I'm wrong - that this backport of
Thank you for the comments, @vltr! I’ll reply in detail once I get to my keyboard later. While I guess Yury might be busy with PyCon and other stuff, let’s wait for his reply a bit later
Approach 1: Yeah I think it has to be like that if we want to support something like this in Python < 3.7 with this backport:
Alternatively, I have two branch thoughts:
Approach 2: Yes that is what worried me - there might be no clean way out. I'll be glad to use immutables in aiocontextvars or vice versa, but it is just a task-local workaround rather than a backport. Therefore I'd prefer to listen to some better ideas and directions here before making further moves. But yeah I agree with you that Python 3.5 and 3.6 deserve a de facto solution to deal with asyncio task locals.
Thanks for your reply, @fantix ! I'll add some more comments on that, I hope you don't mind
I may have a very strong opinion against monkey patching. When
Please, for me this just seems right. I don't think that taking a step backward is bad for a a backport of Python 3.7
Couldn't agree more, hence my comments here. Just to be clear, some of them are just personal opinions and have the sole purpose of giving the point of view of a developer that already use variables bound to tasks within asyncio - even if my approach looks stupidly simple
Yeah I prefer your alternative asyncio module over monkey patching too, if
referenced a pull request that will
Aug 29, 2018
referenced this issue
Nov 12, 2018
My two cents. There is no smell in copying code here - yes, we will have two different copies, but they are for different pythons and their asyncio versions are different anyway. This will probably require maintenance in the form of copying any security fixes if there will be any. This is trivial though. So if code could just be copied as is and won't require adaptation for older python this is a way to go.
Monkey patching is fine too because our targets python 3.5/3.6 asyncio won't change. This will require some hacking and would be heavier on docs - we would have old asyncio with some new features unlike simply new asyncio. Overall looks worse than just copying code.
P.S. asyncio should not have been included into standard lib so early.
Yeah, we just need to copy a lot of code -- event loop & Task/Future implementations (both in Python and C). At this point I think it's easier to just use Python 3.7 (or we can make uvloop work with contextvars on 3.6 & 3.5 if someone is interested to work on the PR).