Can we use pyo3-asyncio with pyo3 >=0.14

[sansyrox]

🐛 Bug Reports

When I am using pyo3-asyncio with pyo3 v0.13.2, I getting an error which was solved in v0.14 . Is there a way to the error below or use the latest pyo3?

🌍 Environment

• Your operating system and version: OSX Big Sur
• Your python version: 3.8.5
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: Pyenv with a local virtualenv
• Your Rust version (rustc --version): 1.52.1
• Your PyO3 version:0.13.2
• Have you tried using latest PyO3 master (replace version = "0.x.y" with git = "https://github.com/awestlake87/pyo3-asyncio")?:

💥 Reproducing

Please provide a minimal working example. This means both the Rust code and the Python.

This is the cargo.toml

[dependencies.pyo3]
version = "0.13.2"
features = ["extension-module"]

[dependencies.pyo3-asyncio]
version = "0.13.4"
features = []

And I get the following error:

error[E0425]: cannot find function `prepare_freethreaded_python` in crate `pyo3`
  --> src/lib.rs:26:11
   |
26 |     pyo3::prepare_freethreaded_python();
   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `pyo3`


error: aborting due to previous error

    Building [=======================> ] 208/209: robyn
For more information about this error, try `rustc --explain E0425`.

error: could not compile `robyn`

To learn more, run the command again with --verbose.
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit code: 101": `cargo rustc --message-format json --manifest-path Cargo.toml --lib -- -C link-arg=-undefined -C link-arg=dynamic_lookup -C link-args=-Wl,-install_name,@rpath/robyn.cpython-38-darwin.so`

[awestlake87]

Sorry for the late response. There is not currently a way to use it with 0.14, the main hold up being #30 right now. Currently, I haven't bumped the pyo3 version to 0.14 on that branch, but I can do that sometime today.

#30 has been approved, but there's still some documentation to be done since it's a fairly major change. You should still be able to use the 0.13 API, but you'll get some deprecation warnings until you start using the newer functions. I expect it'll be at least a day or two before this is released as 0.14.

[awestlake87]

Ok, I've got the version bumped to 0.14 on the init-fixes branch. Change your pyo3-asyncio dependency to something like this in your toml and you should be good to go until the official release:

[dependencies]
pyo3 = "0.14"
pyo3-asyncio = { git = "https://github.com/awestlake87/pyo3-asyncio", branch = "init-fixes" }

[sansyrox]

Thank you @awestlake87! 😄

[awestlake87]

No problem! Let me know if you run into any issues with that branch. You're a bit of a guinea pig for the pyo3-asyncio 0.14 release, so any feedback would be valuable.

[sansyrox]

@awestlake87 absolutely. I am getting some deprecation warnings in the update.

  --> src/lib.rs:25:5
   |
25 |     pyo3_asyncio::try_init(py)?;
   |     ^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default


warning: use of deprecated function `pyo3_asyncio::into_future`: Use pyo3_asyncio::async_std::into_future or pyo3_asyncio::tokio::into_future
   --> src/processor.rs:108:17
    |
108 |                 pyo3_asyncio::into_future(coro?)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^


warning: use of deprecated function `pyo3_asyncio::run_forever`
  --> src/server.rs:15:5
   |
15 | use pyo3_asyncio::run_forever;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^


warning: use of deprecated function `pyo3_asyncio::run_forever`
  --> src/server.rs:66:9
   |
66 |         run_forever(py).unwrap()
   |         ^^^^^^^^^^^


warning: 4 warnings emitted

Do you have any docs that I can refer for these?

[awestlake87]

Yeah, the initialization changes affected the way this library deals with the event loop and this was significant enough to warrant a new set of API functions with slightly different semantics. The old API is still usable, but like you've noticed, it's gated behind some deprecation attributes.

You have a couple of options:

• Add #[allow(deprecated)] to your functions and ignore it for now
• Migrate to the new API even though it's not fully documented yet

I haven't put together a formal migration guide for 0.14 yet, but if you're ready to convert to the 0.14 API, here are some general migration tips:

1. pyo3_asyncio::try_init can be removed
2. pyo3_asyncio::into_future should be replaced with pyo3_asyncio::tokio::into_future
3. pyo3_asyncio::tokio::into_coroutine should be replaced with pyo3_asyncio::tokio::future_into_py
4. pyo3_asyncio::run_forever has been deprecated for 0.14 since it's generally considered an edge case and can be handled via pyo3. It looks like you're using it, so I'd recommend looking at test_async_std_run_forever.rs or test_tokio_multi_thread_run_forever.rs

[sansyrox]

@awestlake87 , removing pyo3_asyncio::try_init and replacing pyo3_asyncio::run_forever with the tokio snippet doesn't make the async loop run.

Earlier, I used to run the snippet:

    pub fn start(&mut self, py: Python, port: u16) {
        if STARTED
            .compare_exchange(false, true, SeqCst, Relaxed)
            .is_err()
        {
            println!("Already running...");
            return;
        }

        let router = self.router.clone();
        let headers = self.headers.clone();

        thread::spawn(move || {
            //init_current_thread_once();
            actix_web::rt::System::new().block_on(async move {
                let addr = format!("127.0.0.1:{}", port);

                HttpServer::new(move || {
                    App::new()
                        .app_data(web::Data::new(router.clone()))
                        .app_data(web::Data::new(headers.clone()))
                        .default_service(web::route().to(index))
                })
                .bind(addr)
                .unwrap()
                .run()
                .await
                .unwrap();
            });
        });

       run_forever(py).unwrap()
    }

Now, I have replaced it with:

    pub fn start(&mut self, py: Python, port: u16) {
        if STARTED
            .compare_exchange(false, true, SeqCst, Relaxed)
            .is_err()
        {
            println!("Already running...");
            return;
        }

        let router = self.router.clone();
        let headers = self.headers.clone();

        thread::spawn(move || {
            //init_current_thread_once();
            actix_web::rt::System::new().block_on(async move {
                let addr = format!("127.0.0.1:{}", port);

                HttpServer::new(move || {
                    App::new()
                        .app_data(web::Data::new(router.clone()))
                        .app_data(web::Data::new(headers.clone()))
                        .default_service(web::route().to(index))
                })
                .bind(addr)
                .unwrap()
                .run()
                .await
                .unwrap();
            });
        });


        let asyncio = py.import("asyncio").unwrap();

        let event_loop = asyncio.call_method0("new_event_loop").unwrap();
        asyncio
            .call_method1("set_event_loop", (event_loop,))
            .unwrap();

        let event_loop_hdl = PyObject::from(event_loop);

        pyo3_asyncio::tokio::get_runtime().spawn(async move {
            tokio::time::sleep(Duration::from_secs(1)).await;

            Python::with_gil(|py| {
                event_loop_hdl
                    .as_ref(py)
                    .call_method1(
                        "call_soon_threadsafe",
                        (event_loop_hdl
                            .as_ref(py)
                            .getattr("stop")
                            // .map_err(|e| dump_err(py, e))
                            .unwrap(),),
                    )
                    // .map_err(|e| dump_err(py, e))
                    .unwrap();
            })
        });

        event_loop.call_method0("run_forever").unwrap();

        println!("test test_run_forever ... ok");
    }

The app closes as soon as it is started. Should I be running an empty loop somewhere?

[awestlake87]

Hmm, I probably cant take a look at this until later this evening, but at first glance nothing seems wrong. Is there more context you can provide? I dont see any conversions anywhere.

[sansyrox]

@awestlake87 , sure. I just replaced the line run_forever(py).unwrap() in the first example with the let asyncio = py.import("asyncio").unwrap();..... line in the second. I have kept everything else the same.

[sansyrox]

Also, if pyo3_asyncio::run_forever has been deprecated. Should I be doing something like pyo3_asyncio::tokio:run_until_complete(loop{}) ?

[awestlake87]

If you're still using the deprecated conversions like into_coroutine, etc then try_init is still necessary. After youve updated the conversions to the 0.14 variants you can remove it. I'm fairly certain the tokio init functions can be removed no problem though

[sansyrox]

@awestlake87 , I got rid of that error. But it is telling me that:
Error: RuntimeError: no running event loop

[awestlake87]

I think that's probably happening in some other code snippet I haven't seen yet. Is this project on GitHub? I'd probably be able to give better advice if I see more parts of it.

[sansyrox]

@awestlake87 , sure. Here is the PR, where I am trying to do the migration: sparckles/Robyn#65

Here is the main repo: https://github.com/sansyrox/robyn