Performance regression of resolving dependencies

[crea]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from fastapi import FastAPI, Depends
from fastapi.testclient import TestClient

app = FastAPI()

Dep0 = Depends(lambda: "dep0")
Dep1 = Depends(lambda: "dep1")
Dep2 = Depends(lambda: "dep2")
Dep3 = Depends(lambda: "dep3")
Dep4 = Depends(lambda: "dep4")
Dep5 = Depends(lambda: "dep5")
Dep6 = Depends(lambda: "dep6")
Dep7 = Depends(lambda: "dep7")
Dep8 = Depends(lambda: "dep8")
Dep9 = Depends(lambda: "dep9")

@app.get("/")
async def read_main(dep0: str = Dep0,
                    dep1: str = Dep1,
                    dep2: str = Dep2,
                    dep3: str = Dep3,
                    dep4: str = Dep4,
                    dep5: str = Dep5,
                    dep6: str = Dep6,
                    dep7: str = Dep7,
                    dep8: str = Dep8,
                    dep9: str = Dep9):
    return {"msg": "Hello World"}


client = TestClient(app)


def test_read_main():
    response = client.get("/")
    assert response.status_code == 200
    assert response.json() == {"msg": "Hello World"}


if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test_read_main()", globals=globals(), number=3000))


# 0.68.2: 12.487527622
# 0.69.0: 21.643492927
# 0.70.0: 21.014488872

Description

The update to fastapi 0.69.0 introduced a performance regression of nearly when using dependencies. The runtime of the sync version with 10 dependencies as shown in the code example increased by around 70%. The runtime of the async version increased by 30%.
This was likely introduced by the starlette upgrade to version 0.15 and their switch to anyio. As a workaround dependencies can be switched to async calls with a smaller performance hit.
Are there any plans to optimize the sync version or to document this behavior?

Operating System

macOS

Operating System Details

No response

FastAPI Version

0.69.0

Python Version

Python 3.9.7

Additional Context

No response

[Kludex]

Oh... This is sad. I did expect a performance regression overall, and it was even mentioned on the Starlette PR that introduced anyio. I did not expect those high numbers here, tho.

@adriangb has been working on performance optimizations on the dependencies' setup, but we might need to find more alternatives anyway...

Would you mind sharing the numbers with an ASGI server instead of the TestClient i.e. uvicorn with an HTTP benchmarking tool?

[adriangb]

We had some discussion on AnyIO's gitter regarding this. I have no idea how to link to a gitter message, so I'll summarize here.

I made a gist comparing anyio to asyncio's performance: https://gist.github.com/adriangb/4769659899abd24f5d184332a2cdbee8

I'd guess the use case you present is even more pathological @crea since not only is async stuff being done, but there's also thread pools and stuff involved.

Quoting @agronholm:

the biggest difference seems to be in that the anyio version does several times more function calls
67821 vs 17890

The TLDR is that AnyIO does considerably more stuff so for small workloads it has quite a bit of overhead. For something like sending an HTTP request, the overhead is negligible. In your test @crea , I suspect that if you add a sleep(1e-4) or something in your dependencies, the difference will disappear.

Regarding my work @Kludex mentioned, I have a proposal in #3641 but that may be a bit ambitious, so I think FastAPI's best concrete bet is #3902 (it doesn't directly address the performance regression you are noticing - in fact it would probably make things worse for the toy use case above, but it does improve performance of the DI system overall).

[crea]

Thank you for your first thoughts on this issue. I attached a comparison using the benchmarking tool hey:
code.txt
fastapi_70.txt
fastapi_68.txt

There the results are not that drastic, just around 16% latency increase.
As mentioned real world performance is a different story - we have seen a latency increase of our application of 30 ms avg. on fastapi 0.68.2 to 40 ms avg. on 0.70.0 on some endpoints where we heavily use (nested) sync dependencies.

[adriangb]

@crea this won't resolve the overall performance regression, but I wonder if we can solve your particular use case? What does your DAG look like? Are the dependencies parallelizable (A -> B and A -> C and A -D, etc. like in your example above) or are they just nested (A -> B -> C)? Would you be willing to abandon FastAPI's DI system?

[crea]

In most instances our dependency graph is shallow with only a few dependences where the depth is 2-3. So there might be the opportunity to parallelize. When it comes to switching the DI system, we really like the simplicity and good integration of fastapi, so having fast resolving out of the box (as before) would be preferred.

[adriangb]

In most instances our dependency graph is shallow with only a few dependences where the depth is 2-3. So there might be the opportunity to parallelize.

Maybe give #3902 a try? It should be a drop-in replacement.

When it comes to switching the DI system, we really like the simplicity and good integration of fastapi, so having fast resolving out of the box (as before) would be preferred.

Unfortunately, I do not see any way of going back for FastAPI. AnyIO would have to improve their overhead.

[adriangb]

One thing that I hadn't noticed until now is that you are using TestClient.
I think TestClient isn't appropriate to benchmark with (it's doing a bunch of complex stuff internally to juggle event loops, lifespans, etc.). I re-wrote your example using httpx instead: https://gist.github.com/adriangb/9615851b30ea107f0d2860f45a56f523#file-fastapi_bench-py

I think the findings are pretty much the same, but at lest we know it's not the TestClient that's slower now.

I think it'd be nice if FastAPI at least gave you the option to disable moving sync dependencies to threads, I'm guessing that's a big part of the performance hit your seeing (moving a sync function that does no IO to a thread is bad, and anyio only exacerbated the problem because it has more overhead than asyncio). I benchmarked this using Xpresso (which does give you that option) and got 4.4 sec (same gist).