Skip to content
master
Go to file
Code

Latest commit

Frank Vieira
Fixed a typo.
0a60e17

Git stats

Files

Permalink
Failed to load latest commit information.

README.md

Last update: 22/06/2018. TLDR: Aiohttp is now using uvloop, Sanic slighly improved .

Before bashing this please open an issue and help us improve it.

Python Web Benchmarks

A simple project to benchmark Python web frameworks against Vibora in a few use cases (more cases will be added soon).

Running Instructions

  1. Install Docker (sudo apt-get install docker.io)

  2. Clone the project (git clone https://github.com/vibora-io/benchmarks)

  3. Build the image (sudo docker build . -t vibora_benchmarks)

  4. Run the benchmark (sudo docker run vibora_benchmarks)

You can configure the benchmark by editing config.json.

Disclaimer

  • Benchmarks are more of a sport than science, don't make assumptions, do your own benchmarks.

  • There are one zillion more important things to consider when choosing a framework. Be wise.

  • I tried to "optimize" each framework but I'm not an expert in every single one.

  • This benchmark mixes sync/async frameworks so to make up a "fair" fight there is no latency between database requests. Async frameworks under high concurrency and high latency databases would perform much better than sync ones thanks to the architecture.

  • Talking about concurrency: WRK is configured to 100 concurrent users which is pretty low number, again, to make it fair against sync frameworks.

  • Users of WSGI are configured to use Gunicorn + Meinheld which is the fastest option although not the most stable one.

Results (AWS c5.2xlarge - 8 CPU cores - Dedicated Tenancy)


Infamous Hello World

Testing how fast they can answer a static response. Not much useful but it sets the high bar for each framework.

Sanic does not send the "Date" http header which is a violation of the HTTP protocol (actually you could argue that this header is not mandatory but since it's used every where by cache engines (and the RFC says unless you don't have a clock you must send) I wouldn't even lose my time arguing).


Frameworks Requests/Sec Version
Tornado 14,197 5.02
Django 22,823 2.0.6
Flask 37,487 1.0.2
Aiohttp 61,252 3.3.2
Sanic 119,764 0.7.0
Vibora 368,456 0.0.6

Validate JSON

Testing how fast they receive a JSON, parse and validate it. Frameworks that do not have a validation engine (or it's too slow) are using Marshmallow, at least for now, unless a fast and tested library appears on the wild.

Vibora/Aiohttp have "streaming enabled" by default which means the user can receive the response in chunks (or even reject it) before sending the response... So beware that Sanic, for example, has a streaming mode which makes the route slower than normal and could affect performance.


Frameworks Requests/Sec Version
Tornado 12,126 5.02
Flask 18,326 1.0.2
Django 18,317 2.0.6
Aiohttp 36,299 3.3.2
Sanic 69,921 0.7.0
Vibora 130,197 0.0.6

Forms Parsing

Testing how fast they receive (and parse) a POST with a multipart-form inside it containing a few parameters and one single file.

Most frameworks with the exception of Vibora and Aiohttp do not have a streaming multipart parser which means they must load the entire form in-memory (big file uploads are impracticable) before letting the user interact with it (unless of course you read the stream yourself and parse it).


Frameworks Requests/Sec Version
Aiohttp 2,315 3.3.2
Django 7,311 2.0.6
Tornado 10,292 5.02
Flask 11,923 1.0.2
Sanic 47,520 0.7.0
Vibora 76,612 0.0.6

Redis API

Testing how fast they receive a POST with a JSON, validate it and do a simple GET in Redis.


Frameworks Requests/Sec Version
Tornado 12,126 5.02
Django 18,317 2.0.6
Flask 18,326 1.0.2
Aiohttp 28,521 3.3.2
Sanic 70,132 0.7.0
Vibora 131,614 0.0.6

About

No description, website, or topics provided.

Resources

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.