Config for running in threadpool even if defined as async def

[adamsuskin]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

@router.post("/example")
def example_endpoint(request: Request) -> None:
    body = request.body() # oh no, starlette wants me to use async/await
    persist_to_db(body) # oh no, my db library wants me to use async/await

# ... this becomes ...

@router.post("/example")
async def example_endpoint(request: Request) -> None:
    body = await request.body()
    await persist_to_db(body) # yay everything is fine, fastapi supports this at the top-level

# in production - wait why is this not running in the thread pool?

Description

Hi! I am coming out of a production outage my company had that ultimately can be sourced back to high volume request bursts on an endpoint defined with async def.

The original author wanted to access request.body() directly which is a coroutine, so they declared their endpoint async as their IDE suggested was required.

Little did they know (or any of us for that matter), that this was the specific difference between having the endpoint run in the main thread (blocking the event loop synchronously) versus running in the thread pool.

In particular, the endpoint would take that body and persist it to the DB and write to GCP Storage (neither of those libraries for us support async/await semantics). Given these interactions, the docs recommend against async def as documented here: https://fastapi.tiangolo.com/async

My feature request is around changing the API for this nuanced difference in functional behavior to an endpoint parameter. In particular, I would imagine the above code example could be written instead like:

@router.post("/example", run_in_threadpool=True)
async def example_endpoint(request: Request) -> None:
    body = await request.body()
    await persist_to_db(body) # yay, I get to use async/await and not think about thread pools

I would actually ask that run_in_threadpool defaults to True, and that the behavior to use the main thread is opt-in explicitly through a configuration key.

It is my understanding (and affirmed by the async docs) that the use case of async def as it stands is very limited because:

• most database libraries/file store libraries don't yet support async/await
• it is incredibly rare to have an endpoint that doesn't have to interact with anything else (the docs call this out too)

Operating System

Linux, Windows, macOS

Operating System Details

No response

FastAPI Version

0.95.2

Pydantic Version

1.10.6

Python Version

3.10.10

Additional Context

No response

[adamsuskin]

Also I didn't say this above, but I would be happy to take a stab at contributing this. I just want to see if I have any mistakes in my understanding of the FastAPI concurrency model / if you would be open to it.

[n8sty]

What you're proposing, if I understand it correctly. seems to me to be a large change to the way that FastAPI operates and there's almost no chance that would be adopted. Many databases do now have async drivers including, from memory so there are likely more than just these, motor, asyncpg, and aiomysql. I'm guessing there are async implementations for interacting with GCP and AWS as well. In general, I'd suggest trying to make your endpoints do as little work as possible and off-load long-running and variable length tasks to BackgroundTasks or some other mechanism that isn't directly within the route.

However, what you can do is run your blocking code in a thread pool. Something like the following:

from starlette.concurrency import run_in_threadpool

@router.post("/example")
async def example_endpoint(request: Request) -> None:
    body = await request.body()
    await run_in_threadpool(persist_to_db, body)
 

[adamsuskin]

Hey @n8sty, appreciate the response! If many libraries do now support async drivers, then I guess that's fine. We may want to update the docs because they suggest otherwise (happy to put up the tiny PR). Forgive me if this is dumb but does that call run_in_threadpool that you made run in the same threadpool that FastAPI is using?