This repository contains Docker images for some of the most used minimal API frameworks available in the market. The goal is to provide reproducible scenarios to compare them against Spdy in the most fair comparison feasible.
- Currently, each contestant must1 provide:
- A router implementation
- A route that returns a 'Hello World!' message in the JSON format.
- Each contestant must have it's own isolated and production-optimized Docker image.
- Each contestant must leverage every programming language/framework optimization avaiable to provide optimal results.
- Each contestant must run in an isolated machine.
- Each contestant must run benchmarks two times, one for warmup(NodeJS V8 and PHP JIT greatly require this) and a second for results.
- Benchmark method can be flexible, but for this repository example, all benchmarks must be ran from inside the container and apply the following technique the using Apache Benchmark tool:
- ab -n 100000 -c 100
- 100K requests with a concurrency level of 100, without Keep-Alive connections
- ab -n 100000 -c 100
- Single worker process per language/framework
Those are the round 1 results, as from 19-07-2022.
| Provider | Digital Ocean |
|---|---|
| Product | CPU Optimized Droplet |
| Image | Docker Version 19.03.12OS Ubuntu 20.04 by Digital Ocean |
| RAM | 8GB |
| CPU | 4 Dedicated CPUs |
| Region | New York 1 |
| PHP 8.1 - Spdy | C++ 17 - uWebsockets | Go 1.18 - Gin | Python 3.9 - FastAPI | NodeJS 16 - Fastify | |
|---|---|---|---|---|---|
| Time taken for tests | 57.129 seconds | 9.208 seconds | 14.929 seconds | 90.420 seconds | 18.484 seconds |
| Complete requests | 100000 | 100000 | 100000 | 100000 | 100000 |
| Failed requests | 0 | 0 | 0 | 0 | 0 |
| Requests per second (mean) | 1750.41 | 10859.95 | 6698.22 | 1105.94 | 5410.07 |
| Time per request (mean) | 57.129 [ms] | 9.208 [ms] | 14.929 [ms] | 90.420 [ms] | 18.484 [ms] |
| Transfer rate | 241.02 [Kbytes/sec] | 901.46 [Kbytes/sec] | 974.64 [Kbytes/sec] | 183.60 [Kbytes/sec] | 887.59 [Kbytes/sec] |
| Percentage of requests served in miliseconds | 50% 57 66% 58 75% 59 80% 60 90% 61 95% 63 98% 65 99% 66 100% 82 (longest request) |
50% 9 66% 10 75% 11 80% 12 90% 13 95% 14 98% 16 99% 18 100% 77 (longest request) |
50% 14 66% 16 75% 18 80% 19 90% 22 95% 25 98% 28 99% 30 100% 68 (longest request) |
50% 88 66% 91 75% 93 80% 95 90% 101 95% 105 98% 110 99% 113 100% 162 (longest request) |
50% 19 66% 20 75% 20 80% 21 90% 23 95% 26 98% 30 99% 33 100% 55 (longest request) |
All images are published on the Github Registry.
Contributions are welcome! If you believe you can improve an image/source or wan'ts to add another contestant, open a pull request!
This work is licensed under the WTFPL license.