Re-introduce thread pool for the sync functions (maybe)

[sansyrox]

#20 (comment)

Read discussion above

And see if introducing a threadpool will impact asgi compliance(most probably not) but still check again.

[JackThomson2]

Using https://docs.rs/tokio/1.7.1/tokio/task/fn.spawn_blocking.html may be the best way, it'll spawn inside tokios internal threadpool and allow you to await it!

[JackThomson2]

Also wondering if it's possible, as the routes are being added we store some meta data about them rust side, i.e. if they're async or not so we don't have to waste time during a request spent inside the GIL which I gather bottlenecks down to 1 thread.

[sansyrox]

@JackThomson2 , I just did that. You can have a look here: #20

[sansyrox]

Thank you for this(https://docs.rs/tokio/1.7.1/tokio/task/fn.spawn_blocking.html) . 😀
I will read more about it.

[JackThomson2]

@JackThomson2 , I just did that. You can have a look here: #20

I suggest we store that info in the rust side, I guess the less time we spend in the GIL the better from what I gather. So each of the hashtables stores an enum of the py function + info about the function. Then a rough idea of what we could do. Something like

let res =  if is_async {
   python_async_fun().await
} else {
  spawn_blocking(move || { python_sync_fun() }).await
}```

Just a rough idea I had in my head :D 

[sansyrox]

@JackThomson2 , we have also have a gitter room in case you want to join (https://gitter.im/robyn_/community#)

I have also added the non blocking sync feature in the latest release. You can have a look 😄

[JackThomson2]

@sansyrox My concern was around the fact that for an async function now we acquire the GIL 3x and for Sync 2x, this is going to bottleneck the server. This can be reduced by one each by storing the async info rust side rather than in a py dict

[sansyrox]

@JackThomson2 , how can you differentiate in rust between sync and async without calling those functions?
I was unable to find any way.

GIL 3x and for Sync 2x

For this, we can do one thing by acquiring gil once and allowing threads inside the code block. But from what I understand, those all are essentially the same things. Do correct me if I am wrong.

[JackThomson2]

@sansyrox When the function is added can't we use the same method? So the router stores get_routes: DashMap<Route, PyFunction>

[sansyrox]

@JackThomson2 , that makes sense.

We would save this gil acquision

    let function: PyFunction = Python::with_gil(|py| {
        let process_object_wrapper: &PyAny = process_object.as_ref(py);
        let py_dict = process_object_wrapper.downcast::<PyDict>().unwrap();
        let is_async: bool = py_dict.get_item("is_async").unwrap().extract().unwrap();
        let handler: &PyAny = py_dict.get_item("handler").unwrap();
        if is_async {
            let coro = handler.call0().unwrap();
            PyFunction::CoRoutine(coro.into())
        } else {
            PyFunction::SyncFunction(handler.into())
        }
    });

on every function call

[sansyrox]

@JackThomson2 , thank you for the feedback. 😄 I'll implement it tomorrow.

But we are still locking it twice here

        PyFunction::CoRoutine(coro) => {
            let x = Python::with_gil(|py| {
                let x = coro.as_ref(py);
                pyo3_asyncio::into_future(x).unwrap()
            });
            let output = x.await.unwrap();
            Python::with_gil(|py| -> PyResult<String> {
                let contents: &str = output.extract(py).unwrap();
                Ok(contents.to_string())
            })
            .unwrap()
        }

Do you have any suggestions for this as well?

[JackThomson2]

I think it looks like a limitation with the py03_asyncio, it may be worth asking the library owned if there's a way to return the result from the async function without having to enter the gil again

[sansyrox]

@JackThomson2 , I did talk a lot with the maintainer(when I was creating robyn initially). I don't think it is possible atm. I'll try asking again.

it may be worth asking the library owned if there's a way to return the result from the async function without having to enter the gil again

Also, it is restricted natively at pyo3 not pyo3-asyncio otherwise it results in a deadlock.

[sansyrox]

Closing this issue as this seems to be complete. @JackThomson2 feel free to reopen if you have anything to add.

Thanks again! 😄