-
Notifications
You must be signed in to change notification settings - Fork 745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
async fn
tracking issue
#1632
Comments
I've been taking a bit of a passive approach to this library lately to see what users need it for, and it seems like these are the main concerns right now. |
For the time being, using |
I've just released pyo3-async (see reddit). Adding a macro for EDIT: This crate is just a one week POC (actually the first time I use PyO3 ever). It lacks advanced tests, documentation examples, benchmarks, etc. I just wanted to explore the event loops interoperation, and because it worked fine, I've written this crate to submit it to your feedback. |
I've run some quick benchmarks to compare both pyo3-async and pyo3-asyncio. Results are as expected:
Here is a simple example illustrating the first point: #[pymodule]
fn example(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_class::<Sender>()?;
Ok(())
}
#[derive(Clone)]
#[pyclass]
struct Sender(tokio::sync::mpsc::Sender<()>);
impl Sender {
async fn future(self) -> PyResult<()> {
self.0.send(()).await.ok();
Ok(())
}
}
#[pymethods]
impl Sender {
#[new]
fn __new__(capacity: usize) -> Self {
let (tx, mut rx) = tokio::sync::mpsc::channel(capacity);
pyo3_asyncio::tokio::get_runtime().spawn(async move { while rx.recv().await.is_some() {} });
Self(tx)
}
fn send_async(&self) -> asyncio::Coroutine {
asyncio::Coroutine::from_future(self.clone().future())
}
fn send_asyncio<'py>(&self, py: Python<'py>) -> PyResult<&'py PyAny> {
pyo3_asyncio::tokio::future_into_py(py, self.clone().future())
}
} import asyncio
import example # built with maturin
import time
CAPA = 100
N = 10000
async def send_async():
sender = example.Sender(CAPA)
start = time.perf_counter()
for _ in range(N):
await sender.send_async()
print("pyo3-async", time.perf_counter() - start)
async def send_asyncio():
sender = example.Sender(CAPA)
start = time.perf_counter()
for _ in range(N):
await sender.send_asyncio()
print("pyo3-asyncio", time.perf_counter() - start)
for _ in range(3):
asyncio.run(send_async())
asyncio.run(send_asyncio())
print("===============") P.S. I'm currently implementing a |
I've publish a new release of pyo3-async with @davidhewitt Would you mind taking a look at this crate? Could it be worth to mention it in PyO3 documentation? |
@wyfo sorry for the slow reply, this looks like a very interesting development! I'm reading through the code today and will let you know what I think ASAP! |
Overall I think it looks like a great crate, though I would suggest you add some CI to test with multiple Python / OS versions. Hopefully our testing in CI here means that you don't have any platform-specific quirks, but I wouldn't guarantee that. This would definitely be welcome of a mention in the PyO3 documentation. I also suspect there are common parts which can be shared with At the moment I think the key API reason which would prevent This prompted me to ask the CPython core devs in the Python discourse how they might want us to deal with these A couple of more specific thoughts:
|
CI is indeed on the TODO list, my first goal was write the code to illustrate the potential of this approach. I don't think however that there are common parts with pyo3-asyncio, because both crates don't operate on the same level, as pyo3 fully relies on runtimes. But both return Python object or trait implementation, so nothing should prevent to work with both crates at the same time. Actually, I did not know about this For the macro part, I've hesitated to do a unique |
Yes, it sounds like if Cython has set a precedent with its Coroutine types it seems reasonable to me for us to do something similar. @adamreichold and I have talked a lot about this kind of problem in the past, see #3073. At the time we were leaning towards a Maybe if we can sketch out a summary of all the pieces that we'd need to add to PyO3 to support Main concerns I'd have would be on:
If it all fits together well, maybe with this latest development we get to a point where @wyfo would you be interested in collecting that together, as this is freshest in your mind? (And if we think it makes sense, implementing? 👀) |
After a second reflexion about this Regarding coroutines, what would be the issue of having one coroutine type per extension? (You have this https://github.com/python/cpython/blob/84b7e9e3fa67fb9b92088d17839d8235f1cec62e/Lib/asyncio/coroutines.py#L38, but it's not really an issue). Coroutines are only meant to be awaited, not used with About
I don't know any performance limitation about this design for now. It is also completely agnostic of any Rust async runtime, and I don't see a reason to change that. In fact, it may be more efficient to spawn a big future into a tokio task, to have it polled exclusively on Rust side, and to wrap the task handle into a Python coroutine, but this optimization is just few lines of code and doesn't require first-class support IMO. But PyO3 needs to expose the Regarding pyo3-asyncio, Here is what I can say for now. For the implementation, I would pretty much copy-paste the content of pyo3-async 😇 |
Thank you, there's a lot of detail here and it's taken me a little while to chew on it. I think you're right that the I'm a little nervous around the complexity of all the different Python async backends. I sort of feel like it's worth us supporting asyncio only in PyO3 itself and then leaving other backends for separate crates, but it depends really on how easy it is to define abstractions which let us do that. As a first step, let's try merging the |
I've written a new POC this weekend based on the current state of #3540, but also #3540 (comment). Here are the improvements compared to the previous implementation:
I've tested this implementation in a real project, awaiting thousands Python awaitables per second inside PyO3 coroutines running on asyncio, and it works well. By the way the performance difference between awaiting My only issue about the default waker implementation for I've also added a lot of tests https://github.com/wyfo/pyo3/blob/async_support/tests/test_coroutine.rs, https://github.com/wyfo/pyo3/blob/async_support/tests/test_awaitable.rs and https://github.com/wyfo/pyo3/blob/async_support/src/gil.rs#L930. |
@davidhewitt should we keep this issue open until all the planned features are merged? |
Would it make sense to make this functionality option/behind a cargo feature? The dependency on |
There was discussion in #3540 (comment) where we decided not to feature gate for the moment, but then we did switch from I wouldn't be opposed to having the feature if it was strongly wanted. |
In fact,
I can remove this dependency in the next PR. |
If itd make sense to implement, rather than depend, that would also work
for my concerns!
…On Wed, Nov 29, 2023, 2:37 AM Joseph Perez ***@***.***> wrote:
In fact, future_util is not really useful.
Here are the uses in the implementation (I'm mixing the current state and
my last POC):
- AtomicWaker and poll_fn for cancellation, poll_fn is trivial to
reimplement, and AtomicWaker could simply be replaced by a mutex, as
there should be no contention issue.
- FutureExt::catch_unwind and CatchUnwind, again trivial to reimplement
- ArcWake and waker, just a mistake of me, I didn't remember about
std::task::Wake, so they can be simply replaced by the std counterpart
I can remove this dependency in the next PR.
—
Reply to this email directly, view it on GitHub
<#1632 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAGBFZX4ICWG6K7Q7SJ4LYG3Q4BAVCNFSM45QJT3ZKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBTGEZTMNJTGM4A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
@wyfo, @davidhewitt Apologies for taking so long to catch up on this thread. I've been taking a look at @wyfo's work in pyo3-async, and it's all positive. I think it's a good approach and clearly has performance benefits over pyo3-asyncio, and I think there's a place for both libraries or a combined approach going forward. pyo3-async is a great candidate for an async library that can be merged into pyo3, and I think it should be the implementation chosen for an async fn for Just a heads up, this'll be a long response! It's taken awhile for me to gather my thoughts because this is a complicated subject. I felt like I needed to dig into the code to really understand the differences and the synergies for these libraries, and to refresh my memory on our past design decisions Past Discussionspyo3-async's design reminds me a lot of @ThibaultLemaire's proof-of-concept for Asyncio-driven futures that we discussed at the start of pyo3-asyncio's development. Reading back through that thread, there were other priorities for pyo3-asyncio at the time, and in hindsight I may have been a bit too dismissive. It seems like now we might have some concrete answers to the concerns that I brought up back then:
Yes! If you're writing Rust code that relies purely on Python's awaitables, then there may not be a need to spawn your future onto a runtime like Tokio. pyo3-async can bridge the gaps between the two languages without the need for a Rust-specific runtime. In addition, if you need a runtime like tokio or async-std for some Rust functionality, you just need a global reference to the runtime, then you can spawn and await the JoinHandle from the pyo3-async function. This is probably good enough in many cases, but not all cases (I'll explain why).
Clearly 😅, sounds like pyo3-async has been blowing pyo3-asyncio out of the water performance-wise. The advantages here were not so clear to us at the time, and we were completely focused on compatibility. As for the hot-path/cold-path, I'm optimistic, but less certain about that. It really depends on the bindings that users need to create. I think we need more users to start using pyo3-async to see how often we can take advantage of these performance wins.
Yes, I think so. Converting these awaitables to Futures that can be polled directly inside Rust and vice-versa has some advantages. The main drawback (as I understand it) is that the Rust Future that's polling an awaitable has to be running on the same thread as the Python event loop in order for some Python functionality like
There was some thought at the time that if Rust futures were polled from asyncio, we could potentially support So can
|
IMO clear docs which can be found easily via doc site or via search engine is necessary, and maybe with some additional examples illustrating some common misunderstandings and their recommended solutions? |
👋 I'm back! Sorry for this quite long period, some health issues kept me away from my keyboard. I'm recovered now and ready to finish the feature. |
@awestlake87 You're right, However, I wonder if By the way, |
As usual, I like writing some POC on the weekend, and #4057 is my latest one. It comes from my previous comment, about the current async support planed not allowing to await Python awaitables in arbitrary threads. This draft PR provides a I know the name |
Maybe the best solution is simply to not expose the type in #3611, but only an async function named Actually, I start liking this |
Welcome back, sorry to hear you were suffering and glad you are better. I see you opened a few PRs already, with apologies I'm a bit busy this week, so I may be slow to review them. I intend to be back at the keyboard as regular soon! |
Something I didn't thought about is to accept |
Hello. I'm not a very experienced Rust developer, I've worked with pyo3 a bit and never with pyo3-asyncio.
These are my dependencies:
Command "cargo add pyo3-asyncio --features tokio" ends up with this:
What am I doing wrong? |
You need to add |
Thank you very much! That's really what I missed. |
Getting this error when i try to use async functions which are using some tokio spawn methods
I am trying to call how do i configure tokio with this experimental async feature? |
Async pyfunctions are just normal pyfunctions (only their return type is converted to coroutine). So you can expose async pyfunction the same way you expose sync pyfunction, using |
See this example from the original POC. You have to manually declare a runtime, like this: fn tokio() -> &'static tokio::runtime::Runtime {
use std::sync::OnceLock;
static RT: OnceLock<tokio::runtime::Runtime> = OnceLock::new();
RT.get_or_init(|| tokio::runtime::Runtime::new().unwrap())
} and you can use it with for example |
the thing is that i am using a crate which is internally calling |
This is the thing you want then https://docs.rs/tokio/latest/tokio/runtime/struct.Runtime.html#method.enter |
Is there any way to create |
See #3613. When coroutine constructor will be exposed, you will be able to return a coroutine from any function/closure, hence using |
FWIW I experienced deadlocks when combining pyo3 async support with tonic. I was running blocks that required tokio (calls into tonic) using the OnceLock of a tokio Runtime suggested elsewhere in the thread. Unfortunately it's difficult for me to provide the code that ends up in this deadlock, but the deadlocks went away when I switched my pyo3 functions back to non-async and just used block_on for any awaits. I also never experienced deadlocks using pyo3-asyncio's tokio::run function which I was using previously. Anyway, I know that isn't anything to go on, and it may be that my code is incorrect somehow, but just want to record this here in case someone else hits the same issue. |
Actually, I've already experienced deadlock in one of my experiment, see #3540 (comment). I think the difference with pyo3-asyncio causing the deadlocks is the fact that |
Anyway, I've discovered today the existence of https://github.com/mozilla/uniffi-rs, and was quite surprised to see it already supports converting a Rust future into a Python awaitable. |
Quite interestingly, at the beginning, uniffi was not mapping a rust future to a python coroutine but directly to a python future; their waker implementation was scheduling rust future poll on the Python event loop, and setting the python future result when the rust future was ready. Then changed their approach to better handle cancellation and now, this is quite similar to what we do, as they return a coroutine yielding a python future for each rust future poll. The main difference is that they use a generated Python coroutine, while we use a pyclass (without taking account PyO3 additional features like Anyway, I find their first approach quite interesting, and think it should have better performance over creating a new future for each poll. I will maybe try to implement it and run some benchmarks to see – I still don't think it will be worth adding this much complexity in the code, but I like making POC. |
Interesting indeed. I think @wyfo I'm keen to review the async PRs again ASAP, I hope to be more available from tomorrow. Which one should I be starting with? I got a bit lost given the time gap; it would indeed be nice to move this further along for 0.22 Also have you seen #4064? I might take a look at that, I guess our protocol methods may need some handling... |
There is no mention of PyO3 in The first PR to review is #3610, then #3611, #3612 and #3613. There is also #4057 draft that can be interesting to complete the support. There are conflicts with recent PRs, so I need to resolve them tonight. I didn't seen #4064. I admit I don't know at all this part of the code (for example, I wasn't able to solve #4113 myself, fortunately @Icxolu was here to help), so I don't know if I will be able to solve the issue myself. I don't really understand why the code is so different between pyfunction/pymethods and magic methods like |
How to call python async function? |
Could someone make a brief rollup post on where async support currently stands in PyO3? This thread is obviously for development, and the user guide is pretty confusing as to exactly what is currently needed to make async fully work. For example, if I enable UPDATE (based on my experience of the last few days):
|
This issue is a placeholder issue which I'd like to use to keep track of what we need to do to support
async fn
for#[pyfunction]
(and#[pymethods]
).While this would be a great feature to have, I think anything which we merge into PyO3 core should be simple to use, performant, and (ideally) async-runtime agnostic.
Users have started researching the design space of how possible cross-language implementations could be written. The most advanced candidate is
pyo3-asyncio
. See the guide as a great example of how to use it: https://pyo3.rs/latest/ecosystem/async-await.htmlAt the moment I suggest we allow some time for that and other crates to mature, and once we understand performance trade-offs / difficulties etc. we can consider upstreaming something here.
As this is supported by third-party crates I think we can afford patience for now.
The text was updated successfully, but these errors were encountered: