Simple Web Benchmark

Table of Contents

• Results
• Testing
  • Using Docker
  • Automation
• Usage
  • Go
  • Crystal
  • Rust
  • Scala
  • Node.js
  • PHP
  • Python
  • Java
  • C++
• Environment

A simple web benchmark of C++, Crystal, Go, Java, Node.js, PHP, Python, Rust and Scala.

Results

Top performers:

Testing

The stats gathered by the hey tool (please run it twice for the JIT optimizations where it’s applicable):

hey -n 50000 -c 256 -t 10 "http://127.0.0.1:3000/"
hey -n 50000 -c 256 -t 10 "http://127.0.0.1:3000/greeting/hello"

Using Docker

Build the image:

$ docker build suite/ -t simple-web-benchmark

Enter the shell in the image:

$ docker run -it --rm -v $(pwd):/src --network="host" simple-web-benchmark

There is a run.sh that could be used to simlify Docker usage:

• ./run.sh build (build the image);

• ./run.sh shell (enter the shell in the image).

Automation

Please use the Rust program to run the tests automatically:

Usage: cargo run --manifest-path suite/Cargo.toml -- [FLAGS] [OPTIONS] <lang>...

Arguments:
  <lang>...  Sets the languages to test ('all' for all)

Options:
  -o, --out [<file>]       Sets an image file to generate (PNG/SVG/TSV are supported) [default: result.svg]
      --verbose <verbose>  Enables the verbose output
  -h, --help               Print help
  -V, --version            Print version

The following languages are supported: crystal, go, nodejs, php, python, rust_actix, rust_hyper_monoio, rust_hyper_tokio, rust_warp, scala.

And another program to get the versions of the languages:

$ cargo run --manifest-path suite/Cargo.toml --bin versions

The same run.sh is used to run those programs (but the commands should be run from within the Docker image shell):

• ./run.sh run (run all tests);

• ./run.sh versions (get the versions of the languages).

Usage

Go

go run go/main.go
go run go/main.go --port=8000

Crystal

Using Crystal:

crystal run --release --no-debug crystal/server.cr
crystal run --release --no-debug crystal/server.cr -- --port=8000

Rust

Please install Nightly Rust.

Sample applications use monoio runtime, hyper HTTP library, Rocket, Tide, warp and Actix web frameworks:

cargo run --manifest-path rust/hyper-tokio/Cargo.toml --release
cargo run --manifest-path rust/hyper-tokio/Cargo.toml --release -- --port=8000

cargo run --manifest-path rust/hyper-monoio/Cargo.toml --release
cargo run --manifest-path rust/hyper-monoio/Cargo.toml --release -- --port=8000

cargo run --manifest-path rust/rocket/Cargo.toml --release
cargo run --manifest-path rust/rocket/Cargo.toml --release -- --port=8000

cargo run --manifest-path rust/tide/Cargo.toml --release
cargo run --manifest-path rust/tide/Cargo.toml --release -- --port=8000

cargo run --manifest-path rust/warp/Cargo.toml --release
cargo run --manifest-path rust/warp/Cargo.toml --release -- --port=8000

cargo run --manifest-path rust/actix-web/Cargo.toml --release
cargo run --manifest-path rust/actix-web/Cargo.toml --release -- --port=8000

Scala

Uses Akka toolkit:

make -C scala clean run
make PORT=8000 -C scala clean run

Node.js

node nodejs/main.js
node nodejs/main.js --port=8000

PHP

Uses standalone web server and Open Swoole extension:

php -q -S 127.0.0.1:3000 php/bare/main.php
php -q -S 127.0.0.1:8000 php/bare/main.php

php -c php/swoole/php.ini php/swoole/main.php
php -c php/swoole/php.ini php/swoole/main.php --port=8000

Python

Uses standalone web server and Twisted engine:

python3 python/main.py
python3 python/main.py --port=8000

pypy3 python/twist.py
pypy3 python/twist.py --port=8000

Please note that CPython has the performance problems running as a standalone server, so we’ve used PyPy3. To install Twisted please use the pip module:

pypy3 -m ensurepip
pypy3 -m pip install twisted

Java

Uses Sprint Boot project:

make -C java clean run
make PORT=8000 -C java clean run

C++

Uses Boost.Beast library:

make -C cpp clean run
make PORT=8000 -C cpp clean run

Environment

CPU: Intel® Xeon® E-2324G, Mem: 16GB DDR4 3200MHz

Base Docker image: Debian GNU/Linux bookworm/sid

Language	Version
Crystal	1.7.2
C++/g++	12.2.0
Go	go1.20.1
Java	19.0.2
Node.js	v19.6.0
PHP	8.2.2
PyPy	7.3.11-final0 for Python 3.9.16
Rust	1.69.0-nightly
Scala	3.2.2