Best practice when a single handler calls both async and sync functions

[Toad2186]

First check

• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.

Description

Consider something like this:

@api.get('/handler')
async def handler():
    ...
    # Slow async function
    await my_async_function()
    ....
    # Slow running sync function
    sync_function()

As written above, it'll work, but the sync function will block the async event loop. Is there any way to avoid this? If the handler is sync instead, then there's no async loop with which I can use to run the my_async_function. Is it possible to get the underlying event loop so I can run my own async function within a synchronous handler? I found a related question--is this my best bet or should I just stick with the above?
#825

[mlaradji]

Hi @Toad2186. One possible solution is to run the slow function in its own thread (https://docs.python.org/3/library/asyncio-eventloop.html#executing-code-in-thread-or-process-pools):

import asyncio
from functools import partial

async def run_in_thread(sync_function, *args, **kwargs):
    # Get the loop that's running the handler.
    loop = asyncio.get_running_loop() 
    # run_in_executor expects a callable with no arguments. We can use `partial` or maybe `lambda` for that.
    sync_function_noargs = partial(sync_function, *args, **kwargs)
    return await loop.run_in_executor(None, sync_function_noargs)

@api.get('/handler')
async def handler():
    ...
    # Slow async function
    await my_async_function()
    ....
    # Slow running sync function is now a coroutine
    await run_in_thread(sync_function)

What's nice about run_in_executor is that it will return the result of the synchronous function, or raise the exception raised by the function. You can do something like:

def slow():
  time.sleep(10)
  raise ValueError

async def main():
  try:
    await run_in_thread(slow)
  except ValueError:
    pass

It can also seamlessly use variables, but care must be taken with non-thread-safe synchronous functions.

[Toad2186]

@mlaradji
Hi, thanks for your reply.

I looked at run_in_executor as you call it currently, it would still run in the async thread, no? Should I create a thread pool and pass that in as the first argument, so that long-running sync functions would block threads in the thread pool instead of blocking the async thread?

[mlaradji]

Hi @Toad2186. That's a good question. If I'm not mistaken, this is what happens in run_in_executor:

1. The sync function is passed to the executor, which will create a new thread to run the function.
2. An async task is added in the event loop, that waits for a result from the executor.
3. The executor waits for the thread to finish, catching its result or exception. It then passes the thread's output to the async task.

No need to create a new pool, as run_in_executor will take care of that if not passed an executor. As the doc says, if None is passed as the executor, the default executor pool would be used, and the pool would be created if it doesn't already exist.

I would suggest to test it out and see if it works as intended. Spawning new threads may also add a bit of overhead.

[euri10]

As written above, it'll work, but the sync function will block the async event loop. Is there any way to avoid this?

from the async def you can run the blocking sync code in an executor like @mlaradji explained or you can use the run_in_threadpool method from starlette which is essentially a wrapper around it, I think it preserves the context on top of running your sync block (dont quote me on that, others may give you a better explanation) but this is pretty much the same.

If the handler is sync instead, then there's no async loop with which I can use to run the my_async_function.

if your handler is a sync one, you're still running in the event loop provided by uvicorn

Is it possible to get the underlying event loop so I can run my own async function within a synchronous handler? I found a related question--is this my best bet or should I just stick with the above?

yes you can but it requires configuring the webserver manually to get the loop, you can find example of that in #825
I'd stick with the above

for a very good sync in sync, sync in async, async in sync and async in async overview this post is excellent

[tiangolo]

There's a utility function for that, run_in_threadpool():

from fastapi.concurrency import run_in_threadpool

@api.get('/handler')
async def handler():
    ...
    # Slow async function
    await my_async_function()
    ....
    # Slow running sync function
    await run_in_threadpool(sync_function)

Sadly it's still not documented, but you can use it already.

[github-actions]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

[mezhaka]

@tiangolo Can we control the size of the thread pool?

[dhodun]

@tiangolo Is this recommended and can it be documented? I had to search around a while to find this (looking to replace something like this from tornado:

  responses = await multi([
      tornado.ioloop.IOLoop.current().run_in_executor(None, get_document_by_id,
                                                      obj["collection_name"],
                                                      obj["doc_id"])
      for obj in parameters
  ])

Use case is running a number of parallel sync DB calls that don't need to block each other. I think what we're looking for is iterate_in_threadpool ?

@euri10 - fantastic post!