-
-
Notifications
You must be signed in to change notification settings - Fork 6.3k
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FastAPI+Uvicorn is running slow than Flask+uWSGI #2690
Comments
Well... you're using "async def" in the FastAPI example when you're doing zeeo asynchronous operations in the endpoint. Try make it a normal function, then re-run the benchmarks. |
For a more detailed benchmark, check TechEmpowers
|
I believe using Uvicorn's Gunicorn worker class along with gunicorn offers more performance than the uvicorn workers |
@dstlny It will be even costly if you do that. |
Because of the overhead of using a threadpool executor for something that takes an instant to run anyways? |
@FalseDev, i didn't get it what you are trying to say here, are you saying that |
If I understood correctly this comment says declaring a very simple sync function (with no waiting) as async might decrease response time. I'm just asking if it's because the overhead of submitting the function to a |
I doubt this will make a big impact on such a small response, but try wrapping the dict you are returning in fastapi in a Did you install uvicorn with uvloop? |
I'm very interesting about this question. Considering the comments show different opinions, I decide to test by myself, there are the testing results:
from fastapi import FastAPI
app = FastAPI(debug=False)
@app.get("/")
async def run():
return {"message": "hello"}
import flask
app = flask.Flask(__name__)
@app.route("/")
def run():
return {"message": "hello"}
These first two testings show the same result as @Arrow-Li wrote at the begining.
from fastapi import FastAPI
app = FastAPI(debug=False)
@app.get("/")
async def run():
return {"message": "hello"}
This is nearly 3x performance than test 1.
from fastapi import FastAPI
app = FastAPI(debug=False)
@app.get("/")
def run():
return {"message": "hello"}
Change
from fastapi import FastAPI
app = FastAPI(debug=False)
@app.get("/")
def run():
return {"message": "hello"}
So, in conclusion, for a function that can be defined as both
|
Wonderful test! So to reach max performance should async + gunicorn |
The answer is no. You should pick the one that fits your case. If you run your ML/DL model in a coroutine (
I think understanding asynchronous I/O a little bit deeper could help, so i'm copying this from one of my answer in Stackoverflow. The question completely depends on what your function does and how it does. Okay, but i need to understand asyncio better.Then assume you have the following code async def x():
a = await service.start()
return a
Note that we were able to do all this because This may not be clear to you at first glance but as I mentioned import asyncio
@asyncio.coroutine
def decorated(x):
yield from x
async def native(x):
await x But these two function are identical does the exact same thing. You can think of But to understand asynchronous I/O deeply we need to have an understanding of what it does and how it does underneath. In most operating systems, a basic API is available with These interfaces enable the user of the API to check whether there is any incoming I/O that should be attended to. For example, your HTTP server wants to check whether any network packets have arrived in order to service them. With the help of this system calls you are able to check this. When we check the manual page of
This gives you a pretty basic idea, and this explains the nature of what asynchronous I/O does. It lets you check whether descriptors can be read and can be written. It makes your code more scalable, by not blocking other things. Your code becomes faster as a bonus, but it is not the actual purpose of asynchronous I/O. So to tidy up. The event loop just keeps yielding, while something is ready. By doing that it does not block. |
Base on @zihaooo test, I post my own result below.
So, fastapi about 2x faster than flask, for me its good but not enough to moving from flask. |
I know async not fit in any cases, what I mean is fastapi advantage compare flask is asgi (dont know if I'm right), if in sync case I rather use flask. |
You can't realistically compare fastapi to flask anyway, as they are intended to do different things. Flask is designed for general websites with no real specialisation, whereas FastAPI has many built in features to specifically aid in the construction of rest-ish APIs. |
That's the main point of the comment i wrote above actually, you can mix up both |
thanks for the benchmarks, because i had the same problem. |
This is another test. Uvicorn Async vs Gunicorn Async with Uvicorn workers. Uvicorn Async1. 17B data
2. 277B data
3. 416KB data
4. 1.49MB data
Gunicorn Async with Uvicorn workers1. 17B data
2. 277B data
3. 416KB data
4. 1.49MB data
The result is interesting. For me, using uvicorn is better. Is there anyone who knows what make this result? |
UvicornConcurrency Level: 10
Time taken for tests: 1.982 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 78500 bytes
HTML transferred: 6500 bytes
Requests per second: 252.21 [#/sec] (mean)
Time per request: 39.649 [ms] (mean)
Time per request: 3.965 [ms] (mean, across all concurrent requests)
Transfer rate: 38.67 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 13 20 24.0 16 284
Processing: 14 19 4.4 18 62
Waiting: 14 18 4.4 18 62
Total: 27 39 25.0 34 300 Concurrency Level: 10
Time taken for tests: 10.315 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 227000 bytes
HTML transferred: 154500 bytes
Requests per second: 48.47 [#/sec] (mean)
Time per request: 206.298 [ms] (mean)
Time per request: 20.630 [ms] (mean, across all concurrent requests)
Transfer rate: 21.49 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 13 21 28.8 17 248
Processing: 34 183 21.4 187 255
Waiting: 34 157 28.4 159 255
Total: 49 204 36.6 205 458 Concurrency Level: 10
Time taken for tests: 67.452 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 269621000 bytes
HTML transferred: 269547000 bytes
Requests per second: 7.41 [#/sec] (mean)
Time per request: 1349.044 [ms] (mean)
Time per request: 134.904 [ms] (mean, across all concurrent requests)
Transfer rate: 3903.53 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 13 46 25.6 40 164
Processing: 911 1291 210.1 1275 2309
Waiting: 134 550 270.0 542 1170
Total: 945 1337 212.9 1305 2339 Concurrency Level: 10
Time taken for tests: 190.623 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 951847500 bytes
HTML transferred: 951773000 bytes
Requests per second: 2.62 [#/sec] (mean)
Time per request: 3812.455 [ms] (mean)
Time per request: 381.245 [ms] (mean, across all concurrent requests)
Transfer rate: 4876.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 13 32 53.1 23 1032
Processing: 1099 3757 1042.8 3732 8124
Waiting: 329 1014 527.3 935 2973
Total: 1128 3789 1043.5 3762 8141 GunicornConcurrency Level: 10
Time taken for tests: 2.011 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 78500 bytes
HTML transferred: 6500 bytes
Requests per second: 248.61 [#/sec] (mean)
Time per request: 40.224 [ms] (mean)
Time per request: 4.022 [ms] (mean, across all concurrent requests)
Transfer rate: 38.12 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 12 20 23.6 16 220
Processing: 13 19 8.6 17 87
Waiting: 13 19 8.6 17 87
Total: 26 39 26.4 33 242 Concurrency Level: 10
Time taken for tests: 3.710 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 227000 bytes
HTML transferred: 154500 bytes
Requests per second: 134.79 [#/sec] (mean)
Time per request: 74.191 [ms] (mean)
Time per request: 7.419 [ms] (mean, across all concurrent requests)
Transfer rate: 59.76 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 12 18 17.8 15 203
Processing: 30 54 26.8 42 206
Waiting: 30 48 21.5 39 206
Total: 43 71 31.5 59 255 Concurrency Level: 10
Time taken for tests: 53.380 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 269621000 bytes
HTML transferred: 269547000 bytes
Requests per second: 9.37 [#/sec] (mean)
Time per request: 1067.594 [ms] (mean)
Time per request: 106.759 [ms] (mean, across all concurrent requests)
Transfer rate: 4932.62 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 12 37 54.0 27 1095
Processing: 325 1021 397.7 936 2650
Waiting: 160 485 208.6 410 1536
Total: 389 1058 400.9 971 2664 Concurrency Level: 10
Time taken for tests: 165.735 seconds
Complete requests: 500
Failed requests: 0
Total transferred: 951847500 bytes
HTML transferred: 951773000 bytes
Requests per second: 3.02 [#/sec] (mean)
Time per request: 3314.692 [ms] (mean)
Time per request: 331.469 [ms] (mean, across all concurrent requests)
Transfer rate: 5608.60 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 13 34 25.8 26 308
Processing: 790 3258 1977.6 2687 11721
Waiting: 380 1091 466.6 1071 4038
Total: 826 3291 1977.9 2712 11736 |
Why not testing with Framework+Redis+Database? Such a simple case can not tell the true. |
@Arrow-Li if you are looking for raw throughput both Flask and FastAPI are limited by WSGI/ASGI servers and framework overhead of course. PyPy can reduce the overhead of Python if used with a server that is made for it. Socketify is a web framework and also provides WSGI and ASGI server Granian is a WSGI and ASGI server If you wanna give a performance boost to existing code go with socketify or granian, socketify performs better for now. FastAPI is more feature complete than pure socketify, so if performance is not the only critical point, you can use FastAPI + socketify ASGI. |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
I'm new to fastapi and I'm trying to test speed between fastapi and flask, but I didn't get a better result by fastapi. pls tell me if I'm making anything wrong?
Example
uvicorn --log-level error --workers 4 fastapi_test:app > /dev/null 2>&1
uwsgi --wsgi-file flask_test.py --process 4 --callable app --http :8000 > /dev/null 2>&1
Result
ab -n 10000 -c 500 http://127.0.0.1:8000/
to test speedEnvironment
Additional context
The text was updated successfully, but these errors were encountered: