Using middleware and sqlalchemy together becomes a problem when there is a load.

[jybaek]

With pool_size(5) and max_overflow(10) as the default, if more than 15/sec loads occur simultaneously with threads(JMeter), the http connection itself is delayed. And there is a timeout in sqlalchemy.

But if you don't use middleware, there is no problem with 100/sec loads at the same time.

First look at the code below. This is an example code of fastapi.

...

app = FastAPI()

app.add_middleware(middleware_class=BaseHTTPMiddleware, dispatch="some_function")


...
# Dependency
def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()
...

@app.get("/users/", response_model=List[schemas.User])
async def read_users(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
    users = crud.get_users(db, skip=skip, limit=limit)
    return users

some_function is simple.

async def some_function(request: Request, call_next):
    response = await call_next(request)
    return response

When using middleware, if the database connection is delayed, why is the http connection affected?

I am so excited to meet fastapi!! thanks. :)

Environment

fastapi==0.63.0
starlette==0.13.6

[FalseDev]

I believe you should use a regular function instead of a regular function while using time taking synchronous components like a db.

As stated here FastAPI (Starlette) runs the async function in the current thread whereas regular functions are ran through an external thread pool.

When you declare a path operation function with normal def instead of async def, it is run in an external threadpool that is then awaited, instead of being called directly (as it would block the server).

In your case the async function is ran in the main thread, hence blocking it from doing anything else.

Simply changing the async function to a regular one will let FastAPI know it could be blocking and should be ran in a different thread

[jybaek]

@FalseDev Thanks for the reply.

My problem is with the use of middleware. If I don't use middleware, there is no problem. And the code above is a sample, so I wrote it wrong. I used await for the DB connection part. e.g.)

users = await crud.get_users(..)
return users

[jybaek]

sorry. It was a personal security tunnel problem.

[tiangolo]

Thanks for the help here @FalseDev ! 👏 🙇

Thanks for reporting back and closing the issue @jybaek 👍

Sorry for the long delay! 🙈 I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.