Creating awaitable methods #50
Comments
|
Yeah, I've gotten this question a few times so it's probably worth adding to the docs. Essentially the problem you're facing is that the
#[pymethods]
impl RustSleeper {
fn sleep<'p>(&self, py: Python<'p>) -> PyResult<&'p PyAny> {
let secs = self.0;
pyo3_asyncio::tokio::future_into_py(py, async move {
tokio::time::sleep(Duration::from_secs(secs)).await;
Python::with_gil(|py| Ok(py.None()))
})
}
}
#[pymethods]
impl RustSleeper {
fn sleep<'p>(this: Py<Self>, py: Python<'p>) -> PyResult<&'p PyAny> {
pyo3_asyncio::tokio::future_into_py(py, async move {
tokio::time::sleep(Duration::from_secs(Python::with_gil(|py| {
this.borrow(py).0
})))
.await;
Python::with_gil(|py| Ok(py.None()))
})
}
}
|
|
Thanks, it's working now. This explanation is very helpful! |
|
Would it be possible to mutate |
|
In a simple case, it can be done with use pyo3::prelude::*;
use tokio::fs::File;
use tokio::io::AsyncReadExt;
#[pyclass]
struct RustSleeper {
#[pyo3(get, set)]
n: u64,
}
async fn get_random_u64() -> u64 {
let mut file = File::open("/dev/urandom").await.unwrap();
file.read_u64().await.unwrap() % 10
}
#[pymethods]
impl RustSleeper {
#[new]
fn new(n: u64) -> Self {
RustSleeper {n}
}
fn change_n(this: Py<Self>, py: Python) -> PyResult<&PyAny> {
pyo3_asyncio::tokio::future_into_py(py, async move {
let n = get_random_u64().await;
Python::with_gil(|py| {
let cell: &PyCell<RustSleeper> = this.as_ref(py);
let mut slf = cell.try_borrow_mut().unwrap();
slf.n = n;
Ok(py.None())
})
})
}
}
#[pymodule]
fn sleeper_rs(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_class::<RustSleeper>()?;
Ok(())
}And use like: import asyncio
import sleeper_rs
sl = sleeper_rs.RustSleeper(2)
async def main():
await sl.change_n()
print(sl.n) # 2
asyncio.run(main())
print(sl.n) # from 0 to 9But I couldn't find a way to use method like this: impl RustSleeper {
async fn async_set_n(&mut self, n: u64) {
self.n = n;
}
}inside |
|
Yeah the example that @1frag provided should work. Another alternative that I've used in my projects is to create an inner structure protected with an I'm away from my computer right now, but I can provide some snippets to elaborate later if needed. |
|
Thanks, that worked great! |
Here's an example for those (like me) that came here to find one: use pyo3::prelude::*;
use std::sync::Arc;
use tokio::sync::Mutex;
struct MyType {
value: bool
}
impl MyType {
pub async fn my_async_fn(&self, or: bool) -> bool {
self.value | or
}
}
#[pyclass(name = "MyType")]
struct PyMyType(Arc<Mutex<MyType>>);
#[pymethods]
impl PyMyType {
#[new]
pub fn new() -> Self {
Self(Arc::new(Mutex::new(MyType { value: false })))
}
pub fn my_async_fn<'a>(&self, py: Python<'a>, or: bool) -> PyResult<&'a PyAny> {
let inner = self.0.clone();
pyo3_asyncio::tokio::future_into_py(py, async move {
Ok(inner.lock().await.my_async_fn(or).await)
})
}
} |
|
@RoastVeg thank you very much for posting this detailed solution! I tried it in my case, which is similar (though slightly more complex). If I don't define a clear where If I do define an explicit lifetime, the compiler now says Maybe it's because the return type of the inner async function is not a simple type that implements Any ideas? |
|
@igiloh-pinecone can you share your function? From what you're saying, I think you're capturing a value with the lifetime of the GIL in your async block (which requires 'static), but without context I couldn't really tell you where + how to get around the error. |
|
@awestlake87 thank you, and apologies for the delayed response! The code itself is slightly convoluted, involving a cascade of wrapped struct method calls. | pub fn upsert_async<'a>(&mut self, py: Python<'a>, vectors: Vec<pinecone_core::Vector>, namespace: &'a str) -> PyResult<&'a PyAny>{
| -- -- `py` is a reference that is only valid in the associated function body
| |
| lifetime `'a` defined here
| let mut inner_index = self.inner.clone();
| / pyo3_asyncio::tokio::future_into_py(py, async move {
| | let res = inner_index.lock().await.upsert(&namespace, &vectors, None).await?;
| | Ok(res)
| | // Ok(Python::with_gil(|py| res.into_py(py)))
| |
| | })
| | ^
| | |
| |_______________`py` escapes the associated function body here
| argument requires that `'a` must outlive `'static`I then suspected that maybe Thank you again for taking the time and effort to look into this problem! |
|
I think your issue is You can do this by converting it to a Rust |
|
@awestlake87 thank you very much, you're absolutely right! However, I only now realized there's a deeper problem with this approach. I ended up simply cloning |
|
From what I can see, your mutex is locked within the async block, so the bigger question IMO is whether you can perform Your function can be called multiple times from python without blocking because it essentially just schedules the task on tokio. The lock occurs inside the async block so these scheduled tasks will run at the same time, but they will wait on each other so that only one is doing the upsert at any given time. If your struct can perform upserts concurrently there's no need for the mutex and you might be able to get away with just an |
Hi, I haven't found any information on how to write asynchronous methods. e.g. I'd like to use module like:
I write the next rust code:
It doesn't work. The compiler's output:
Pretty understandable, but I cannot set
'staticforselfdue to specific handling in pymethods:Error when set `'static` for `self`
or (with `-Z macro-backtrace`)
Dependencies:
So, is it possible to write async method? And if yes, maybe extend the documentation or examples of this library?
The text was updated successfully, but these errors were encountered: