This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
- Helping decide beetween languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0
ℹ️ you need wrk stable
git clone --branch 4.1.0 https://github.com/wg/wrkdocker is used for development purpose, production results will be computed on DigitalOcean
- Install all dependencies
shards install- Build internal tools
shards build- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Start the benchmark ....
tools is a list of language / framework to challenge (example : ruby kemal amber go python)
bin/benchmarker [tools]Last update: 2019-02-18
OS: Linux (version: 4.16.3-301.fc28.x86_64, arch: x86_64)
CPU Cores: 8
threads: 9, connections: 1000
1️⃣ nickel (rust)
2️⃣ roda (ruby)
3️⃣ rack-routing (ruby)
4️⃣ zend-expressive (php)
5️⃣ laravel (php)
| Language (Runtime) | Framework (Middleware) | Average | 50th percentile | 90th percentile | 99th percentile | 99.9th percentile | Standard deviation |
|---|---|---|---|---|---|---|---|
| rust (1.32) | nickel (0.11) | 0.08 ms | 0.08 ms | 0.11 ms | 0.14 ms | 0.85 ms | 30.67 |
| ruby (2.6) | roda (3.17) | 3.50 ms | 0.20 ms | 12.62 ms | 29.57 ms | 84.89 ms | 6682.67 |
| ruby (2.6) | rack-routing (0.0) | 4.25 ms | 0.22 ms | 15.54 ms | 33.77 ms | 85.56 ms | 7859.33 |
| php (7.3) | zend-expressive (3.2) | 109.70 ms | 0.32 ms | 215.30 ms | 2497.90 ms | 6779.62 ms | 433080.33 |
| php (7.3) | laravel (5.7) | 109.22 ms | 0.32 ms | 263.79 ms | 1921.54 ms | 6874.03 ms | 404417.67 |
| php (7.3) | zend-framework (3.1) | 151.99 ms | 0.33 ms | 281.59 ms | 3220.11 ms | 7013.90 ms | 562038.67 |
| php (7.3) | slim (3.12) | 133.17 ms | 0.33 ms | 273.99 ms | 2597.47 ms | 5995.01 ms | 468965.33 |
| php (7.3) | lumen (5.7) | 137.61 ms | 0.33 ms | 262.62 ms | 2950.84 ms | 6827.30 ms | 521257.67 |
| ruby (2.6) | flame (4.18) | 6.57 ms | 0.38 ms | 22.19 ms | 46.10 ms | 106.43 ms | 10912.33 |
| rust (1.32) | iron (0.6) | 0.38 ms | 0.39 ms | 0.63 ms | 0.86 ms | 7.35 ms | 210.00 |
| ruby (2.6) | hanami (1.3) | 7.46 ms | 0.42 ms | 24.99 ms | 50.66 ms | 154.45 ms | 12270.67 |
| ruby (2.6) | sinatra (2.0) | 8.55 ms | 0.58 ms | 26.47 ms | 51.29 ms | 129.14 ms | 12681.67 |
| php (7.3) | symfony (4.2) | 162.24 ms | 1.13 ms | 339.80 ms | 2992.53 ms | 5765.67 ms | 539357.33 |
| rust (nightly) | rocket (0.4) | 53.31 ms | 1.56 ms | 3.43 ms | 1750.69 ms | 5506.26 ms | 344884.00 |
| python (3.7) | japronto (0.1) | 3.10 ms | 2.29 ms | 6.26 ms | 13.90 ms | 44.65 ms | 2968.00 |
| c (11) | agoo-c (0.3) | 2.96 ms | 2.30 ms | 5.60 ms | 12.26 ms | 50.31 ms | 2619.33 |
| rust (1.32) | actix-web (0.7) | 3.17 ms | 2.32 ms | 6.43 ms | 14.07 ms | 145.60 ms | 3836.00 |
| python (3.6) | vibora (0.0) | 4.09 ms | 3.24 ms | 8.56 ms | 15.83 ms | 49.84 ms | 3521.67 |
| ruby (2.6) | rails (5.2) | 34.29 ms | 3.25 ms | 115.22 ms | 308.58 ms | 886.72 ms | 65880.33 |
| go (1.11) | fasthttprouter (0.1) | 3.83 ms | 3.33 ms | 5.98 ms | 12.06 ms | 51.16 ms | 2383.33 |
| crystal (0.27) | spider-gazelle (1.4) | 4.14 ms | 3.58 ms | 7.85 ms | 14.77 ms | 30.82 ms | 2991.33 |
| cpp (11.0) | evhtp (1.2) | 3.48 ms | 3.63 ms | 5.48 ms | 8.07 ms | 83.57 ms | 2201.33 |
| crystal (0.27) | onyx (0.5) | 4.02 ms | 3.74 ms | 6.92 ms | 13.51 ms | 56.72 ms | 2940.00 |
| rust (1.32) | gotham (0.3) | 4.49 ms | 3.84 ms | 8.39 ms | 18.88 ms | 192.32 ms | 4741.67 |
| nim (0.19) | jester (0.4) | 4.59 ms | 4.03 ms | 7.24 ms | 12.60 ms | 33.13 ms | 2495.00 |
| ruby (2.6) | agoo (2.7) | 6.33 ms | 4.17 ms | 10.30 ms | 73.37 ms | 122.88 ms | 11062.33 |
| node (11.9) | restana (2.8) | 5.50 ms | 4.65 ms | 9.02 ms | 17.12 ms | 235.50 ms | 6174.67 |
| node (11.9) | polka (0.5) | 6.34 ms | 4.83 ms | 9.57 ms | 19.62 ms | 318.15 ms | 10010.67 |
| node (11.9) | rayo (1.2) | 6.18 ms | 4.88 ms | 9.63 ms | 18.15 ms | 264.39 ms | 7505.33 |
| c (99) | kore (3.1) | 5.37 ms | 5.30 ms | 8.96 ms | 11.37 ms | 37.24 ms | 2764.33 |
| go (1.11) | muxie (1.0) | 6.34 ms | 5.31 ms | 10.42 ms | 21.53 ms | 157.87 ms | 4505.67 |
| go (1.11) | iris (11.1) | 6.43 ms | 5.37 ms | 10.55 ms | 21.23 ms | 147.26 ms | 4337.67 |
| java (8) | act (1.8) | 6.42 ms | 5.58 ms | 10.84 ms | 20.27 ms | 160.11 ms | 4191.67 |
| go (1.11) | chi (4.0) | 7.06 ms | 5.65 ms | 11.36 ms | 23.34 ms | 235.01 ms | 7459.67 |
| csharp (7.3) | aspnetcore (2.2) | 6.51 ms | 5.74 ms | 9.53 ms | 17.17 ms | 216.48 ms | 5319.00 |
| go (1.11) | gin (1.3) | 7.16 ms | 5.81 ms | 11.99 ms | 24.23 ms | 118.06 ms | 5079.33 |
| go (1.11) | echo (4.0) | 7.28 ms | 5.81 ms | 11.97 ms | 24.57 ms | 186.05 ms | 7184.00 |
| go (1.11) | beego (1.11) | 7.30 ms | 5.97 ms | 12.08 ms | 24.70 ms | 216.23 ms | 5646.00 |
| node (11.9) | fastify (1.14) | 8.08 ms | 6.15 ms | 12.00 ms | 30.19 ms | 403.54 ms | 12484.00 |
| node (11.9) | foxify (0.10) | 7.78 ms | 6.31 ms | 11.32 ms | 22.14 ms | 309.41 ms | 9849.33 |
| go (1.11) | gorilla-mux (1.7) | 7.92 ms | 6.38 ms | 12.67 ms | 25.66 ms | 294.57 ms | 8962.00 |
| python (3.7) | falcon (1.4) | 8.13 ms | 6.73 ms | 13.17 ms | 24.26 ms | 216.62 ms | 7490.33 |
| node (11.9) | koa (2.7) | 7.98 ms | 6.80 ms | 12.09 ms | 23.53 ms | 280.53 ms | 8848.00 |
| scala (2.12) | akkahttp (10.1) | 157.99 ms | 7.16 ms | 24.43 ms | 4048.65 ms | 6955.29 ms | 690723.33 |
| node (11.9) | muneem (2.4) | 9.66 ms | 7.25 ms | 15.65 ms | 35.52 ms | 440.64 ms | 15439.67 |
| python (3.7) | bottle (0.12) | 9.54 ms | 8.43 ms | 15.17 ms | 32.13 ms | 220.61 ms | 7916.67 |
| node (11.9) | restify (7.7) | 9.27 ms | 8.60 ms | 12.67 ms | 21.85 ms | 238.66 ms | 6250.33 |
| go (1.11) | gf (1.5) | 9.74 ms | 8.75 ms | 13.47 ms | 30.63 ms | 306.74 ms | 10217.67 |
| node (11.9) | express (4.16) | 10.95 ms | 8.82 ms | 15.14 ms | 37.25 ms | 483.50 ms | 18552.67 |
| python (3.7) | hug (2.4) | 14.24 ms | 10.14 ms | 29.53 ms | 59.69 ms | 220.31 ms | 12214.33 |
| python (3.7) | starlette (0.10) | 12.48 ms | 10.53 ms | 23.33 ms | 37.69 ms | 66.44 ms | 7742.00 |
| swift (4.2) | perfect (3.0) | 11.01 ms | 11.09 ms | 12.99 ms | 15.14 ms | 45.59 ms | 1730.67 |
| swift (4.2) | vapor (3.0) | 18.54 ms | 11.59 ms | 21.39 ms | 196.18 ms | 1230.09 ms | 56357.67 |
| scala (2.12) | http4s (0.18) | 14.11 ms | 12.35 ms | 25.34 ms | 44.95 ms | 566.41 ms | 14360.00 |
| python (3.7) | aiohttp (3.5) | 21.18 ms | 17.87 ms | 43.48 ms | 64.07 ms | 99.96 ms | 14398.67 |
| node (11.9) | hapi (18.1) | 26.42 ms | 18.92 ms | 31.56 ms | 246.47 ms | 1014.28 ms | 52397.33 |
| swift (4.2) | kitura (2.5) | 28.54 ms | 21.36 ms | 38.16 ms | 203.37 ms | 926.82 ms | 46721.00 |
| crystal (0.27) | router.cr (0.2) | 26.01 ms | 25.19 ms | 33.14 ms | 41.93 ms | 234.27 ms | 7419.67 |
| python (3.7) | flask (1.0) | 31.38 ms | 25.42 ms | 50.06 ms | 104.47 ms | 644.23 ms | 29052.67 |
| crystal (0.27) | raze (0.3) | 28.74 ms | 29.53 ms | 37.57 ms | 57.75 ms | 245.71 ms | 10698.33 |
| crystal (0.27) | kemal (0.25) | 28.39 ms | 30.16 ms | 37.24 ms | 42.89 ms | 251.09 ms | 10992.00 |
| crystal (0.27) | amber (0.11) | 32.56 ms | 31.01 ms | 43.54 ms | 51.10 ms | 242.91 ms | 9602.00 |
| crystal (0.27) | lucky (0.11) | 33.11 ms | 32.83 ms | 40.61 ms | 48.20 ms | 327.12 ms | 11622.33 |
| crystal (0.27) | orion (1.6) | 36.26 ms | 34.26 ms | 46.32 ms | 53.59 ms | 253.71 ms | 9522.67 |
| python (3.7) | sanic (18.12) | 41.72 ms | 34.48 ms | 76.44 ms | 132.38 ms | 269.33 ms | 26635.67 |
| python (3.7) | quart (0.8) | 51.14 ms | 42.45 ms | 98.34 ms | 148.46 ms | 296.25 ms | 30674.33 |
| python (3.7) | django (2.1) | 49.31 ms | 43.81 ms | 80.34 ms | 110.02 ms | 472.19 ms | 23494.33 |
| python (3.7) | responder (1.1) | 67.94 ms | 61.32 ms | 116.83 ms | 153.69 ms | 203.53 ms | 32925.00 |
| python (3.7) | tornado (5.1) | 75.64 ms | 74.00 ms | 93.22 ms | 108.81 ms | 553.48 ms | 22206.33 |
1️⃣ (agoo-c) (c)
2️⃣ (japronto) (python)
3️⃣ (actix-web) (rust)
4️⃣ (vibora) (python)
5️⃣ (evhtp) (cpp)
| Language (Runtime) | Framework (Middleware) | Requests / s | Throughput |
|---|---|---|---|
| c (11) | agoo-c (0.3) | 337630.33 | 195.29 MB |
| python (3.7) | japronto (0.1) | 326326.00 | 390.49 MB |
| rust (1.32) | actix-web (0.7) | 316372.00 | 359.64 MB |
| python (3.6) | vibora (0.0) | 262823.67 | 298.49 MB |
| cpp (11.0) | evhtp (1.2) | 252369.33 | 245.04 MB |
| go (1.11) | fasthttprouter (0.1) | 243958.00 | 393.80 MB |
| crystal (0.27) | spider-gazelle (1.4) | 239767.67 | 256.79 MB |
| crystal (0.27) | onyx (0.5) | 237430.33 | 223.45 MB |
| rust (1.32) | gotham (0.3) | 226906.33 | 465.03 MB |
| nim (0.19) | jester (0.4) | 225660.67 | 453.83 MB |
| ruby (2.6) | agoo (2.7) | 214335.67 | 123.93 MB |
| java (8) | act (1.8) | 205410.00 | 401.20 MB |
| node (11.9) | restana (2.8) | 179972.67 | 269.61 MB |
| node (11.9) | polka (0.5) | 165473.33 | 247.73 MB |
| node (11.9) | rayo (1.2) | 163475.67 | 244.58 MB |
| rust (1.32) | iron (0.6) | 157268.00 | 197.86 MB |
| go (1.11) | muxie (1.0) | 155074.67 | 207.67 MB |
| go (1.11) | iris (11.1) | 151878.00 | 203.46 MB |
| csharp (7.3) | aspnetcore (2.2) | 145783.00 | 237.63 MB |
| go (1.11) | chi (4.0) | 142837.67 | 190.65 MB |
| node (11.9) | fastify (1.14) | 141669.33 | 336.27 MB |
| go (1.11) | gin (1.3) | 139111.33 | 244.19 MB |
| go (1.11) | echo (4.0) | 137138.33 | 240.86 MB |
| go (1.11) | beego (1.11) | 135660.00 | 182.80 MB |
| node (11.9) | foxify (0.10) | 132717.67 | 278.87 MB |
| node (11.9) | koa (2.7) | 127584.33 | 269.83 MB |
| go (1.11) | gorilla-mux (1.7) | 127152.67 | 170.33 MB |
| python (3.7) | falcon (1.4) | 122912.67 | 315.59 MB |
| node (11.9) | muneem (2.4) | 122079.33 | 182.89 MB |
| python (3.7) | bottle (0.12) | 107290.00 | 264.34 MB |
| go (1.11) | gf (1.5) | 105979.33 | 161.10 MB |
| node (11.9) | restify (7.7) | 105661.33 | 185.30 MB |
| rust (1.32) | nickel (0.11) | 105398.67 | 208.80 MB |
| node (11.9) | express (4.16) | 102183.00 | 250.02 MB |
| swift (4.2) | perfect (3.0) | 89548.33 | 84.18 MB |
| python (3.7) | starlette (0.10) | 83376.67 | 179.61 MB |
| scala (2.12) | http4s (0.18) | 77559.00 | 135.72 MB |
| python (3.7) | hug (2.4) | 77199.33 | 191.48 MB |
| swift (4.2) | vapor (3.0) | 73548.00 | 123.42 MB |
| scala (2.12) | akkahttp (10.1) | 73314.00 | 157.12 MB |
| php (7.3) | zend-expressive (3.2) | 67766.00 | 335.72 MB |
| php (7.3) | slim (3.12) | 65443.33 | 324.08 MB |
| php (7.3) | symfony (4.2) | 64640.00 | 320.83 MB |
| php (7.3) | zend-framework (3.1) | 63351.67 | 313.91 MB |
| php (7.3) | lumen (5.7) | 60082.00 | 312.46 MB |
| c (99) | kore (3.1) | 58908.67 | 159.62 MB |
| rust (nightly) | rocket (0.4) | 56745.33 | 89.52 MB |
| php (7.3) | laravel (5.7) | 52600.67 | 274.29 MB |
| python (3.7) | aiohttp (3.5) | 50970.33 | 115.64 MB |
| node (11.9) | hapi (18.1) | 49230.67 | 128.04 MB |
| swift (4.2) | kitura (2.5) | 40450.67 | 74.99 MB |
| crystal (0.27) | router.cr (0.2) | 38722.67 | 36.29 MB |
| ruby (2.6) | roda (3.17) | 36699.00 | 34.95 MB |
| crystal (0.27) | kemal (0.25) | 35502.00 | 57.83 MB |
| crystal (0.27) | raze (0.3) | 35322.33 | 33.09 MB |
| python (3.7) | flask (1.0) | 33049.33 | 81.37 MB |
| crystal (0.27) | lucky (0.11) | 30500.00 | 37.36 MB |
| crystal (0.27) | amber (0.11) | 30371.00 | 55.43 MB |
| ruby (2.6) | rack-routing (0.0) | 30152.67 | 17.40 MB |
| crystal (0.27) | orion (1.6) | 27008.67 | 44.01 MB |
| python (3.7) | sanic (18.12) | 25190.00 | 44.87 MB |
| python (3.7) | quart (0.8) | 20378.00 | 40.59 MB |
| python (3.7) | django (2.1) | 19895.67 | 57.77 MB |
| ruby (2.6) | flame (4.18) | 19487.67 | 11.27 MB |
| ruby (2.6) | hanami (1.3) | 17093.33 | 129.20 MB |
| ruby (2.6) | sinatra (2.0) | 14992.00 | 38.90 MB |
| python (3.7) | responder (1.1) | 14689.67 | 32.02 MB |
| python (3.7) | tornado (5.1) | 12900.33 | 38.17 MB |
| ruby (2.6) | rails (5.2) | 3735.67 | 11.43 MB |
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
- Taichiro Suzuki - Author, maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Mainainer