Benchmarking test - statistical results

[OvermindDL1]

So my new server is setup, nice and empty and ready to run tests on (for now), so I'm playing with it. :-)

8-core (16-hyperthreads), 3.7ghz, 16gigs ram (for now), etc... etc...

First, the current result sets that are currently built into this git project are not indicative of the actual through-put that a server can sustain as it always tests a simple iteration of command and how fast they complete rather then how long each request takes, the average, the longest a request took, etc... etc...

So I whipped up a quick script to test the statistical parts in far far greater detail than this current git HEAD does, these are the results for a set of servers. I used rust, crystal, go, python, elixir, and node for the servers, mostly because I haven't installed (or figured out how, depending) the rest of the stuff, plus these run the fastest overall. I did have an issue with server_python_japronto, it refused to run... The server_python_flask did not install flask, I had to do that manually, that should be fixed... japronto does not seem to be in pip, so it remained missing. I'll post the results here as I get them (it outputs markdown format so the posts will be a run with a bit of description at the top of each).

[OvermindDL1]

It's quite fun watching all 16 hyperthreads get pegged to 100% I have to say. ^.^

First 'big' run was done at just 1s warmup and 1s test run time, so it is likely not accurate but is a quick way to make sure it is all working at least, the results:

Total Cores: 16
Processing servers:

• bin/server_rust_iron
• bin/server_rust_nickel
• bin/server_rust_rocket
• bin/server_crystal_kemal
• bin/server_crystal_raze
• bin/server_crystal_router_cr
• bin/server_go_echo
• bin/server_go_fasthttprouter
• bin/server_go_gin
• bin/server_go_gorilla_mux
• bin/server_go_iris
• bin/server_python_flask
• bin/server_python_sanic
• bin/server_elixir_phoenix
• bin/server_elixir_plug
• bin/server_node_clusterexpress
• bin/server_node_express

Processing: bin/server_rust_iron

• • Launching server: bin/server_rust_iron
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/26450118710993766957024200955
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/52203959857881845450048080981
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_rust_iron

Processing: bin/server_rust_nickel

• • Launching server: bin/server_rust_nickel
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/74778458944759186539529287107
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/54696973881595090384887372560
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_rust_nickel

Processing: bin/server_rust_rocket

• • Launching server: bin/server_rust_rocket
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/97949065908733122689584009393
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/26743958305840587842268905169
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_rust_rocket

Processing: bin/server_crystal_kemal

• • Launching server: bin/server_crystal_kemal
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/29743438573675434744681532462
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/19056941744999552588288086747
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_crystal_kemal

Processing: bin/server_crystal_raze

• • Launching server: bin/server_crystal_raze
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/30184765330408810171274973053
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/96302508365455581451705095958
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_crystal_raze

Processing: bin/server_crystal_router_cr

• • Launching server: bin/server_crystal_router_cr
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/93352312178209284092014350375
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/84091821568116489722588130204
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_crystal_router_cr

Processing: bin/server_go_echo

• • Launching server: bin/server_go_echo
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/3788566778558216355877466848
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/54409751513928336873406361495
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_go_echo

Processing: bin/server_go_fasthttprouter

• • Launching server: bin/server_go_fasthttprouter
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/82231991195676397195429489733
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/59471051778010463301767929231
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_go_fasthttprouter

Processing: bin/server_go_gin

• • Launching server: bin/server_go_gin
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/73952613769885501160680666196
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/84382352193437242343512219554
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_go_gin

Processing: bin/server_go_gorilla_mux

• • Launching server: bin/server_go_gorilla_mux
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/87477833345627742276627299725
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/45743836668827106832766751597
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_go_gorilla_mux

Processing: bin/server_go_iris

• • Launching server: bin/server_go_iris
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/85737280622104170131907729865
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/84834162416600765720412354312
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_go_iris

Processing: bin/server_python_flask

• • Launching server: bin/server_python_flask
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/94792907760373159778033179283
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/90734747551170605875849811761
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_python_flask

Processing: bin/server_python_sanic

• • Launching server: bin/server_python_sanic
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/65354821648575277137885277394
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/47219882845453329028649374101
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_python_sanic

Processing: bin/server_elixir_phoenix

• • Launching server: bin/server_elixir_phoenix
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/1393429505641519114662876030
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/74989099520657335853701921860
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_elixir_phoenix

Processing: bin/server_elixir_plug

• • Launching server: bin/server_elixir_plug
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/2028900076188986538076604035
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/30198852409522963948200354436
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_elixir_plug

Processing: bin/server_node_clusterexpress

• • Launching server: bin/server_node_clusterexpress
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/45547655671309327389192136422
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/30258168433532889024502313449
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_node_clusterexpress

Processing: bin/server_node_express

• • Launching server: bin/server_node_express
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/67498271887158652849986996005
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/75041962912088020115693004293
• • Performing test for 1s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 1s...
• • Test Complete

Shutting down server: bin/server_node_express

Path	URL	Errors	Total Requests Count	Total Requests Throughput	Req/s Avg	Req/s Stdev	Req/s Max	Req/s +/-	Latency Avg	Latency Stdev	Latency Max	Latency +/-	50%	75%	90%	99%
bin/server_crystal_kemal	http://127.0.0.1:3000/	0	65104	6.71MB	8.18k	1.18k	10.10k	62.50%	13.82ms	3.82ms	42.02ms	79.28%	12.14ms	12.66ms	19.24ms	28.70ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user	0	49889	5.14MB	6.27k	533.68	7.08k	70.00%	19.30ms	4.86ms	38.67ms	91.19%	22.41ms	23.00ms	23.70ms	33.62ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user/0	0	45991	6.05MB	5.78k	574.51	6.32k	90.00%	21.52ms	5.22ms	48.83ms	95.72%	24.77ms	25.23ms	25.61ms	36.72ms
bin/server_crystal_raze	http://127.0.0.1:3000/	0	80060	4.73MB	10.06k	450.00	10.74k	63.75%	11.49ms	2.48ms	19.01ms	83.38%	10.49ms	10.61ms	16.89ms	17.64ms
bin/server_crystal_raze	http://127.0.0.1:3000/user	0	69163	4.09MB	8.68k	1.21k	10.06k	73.75%	13.45ms	4.37ms	212.16ms	78.31%	11.48ms	17.70ms	18.17ms	21.10ms
bin/server_crystal_raze	http://127.0.0.1:3000/user/0	0	64250	5.64MB	8.06k	784.72	8.97k	68.75%	14.39ms	3.96ms	219.08ms	69.81%	12.51ms	18.50ms	18.74ms	21.53ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/	0	77489	4.58MB	9.74k	1.83k	19.10k	75.00%	11.34ms	3.79ms	211.21ms	77.76%	9.75ms	14.92ms	15.52ms	22.70ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user	0	70123	4.15MB	8.82k	2.02k	12.55k	56.25%	10.56ms	3.55ms	212.93ms	96.15%	8.59ms	13.42ms	13.72ms	15.98ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user/0	0	65843	5.78MB	8.27k	1.00k	10.06k	61.25%	13.19ms	2.92ms	22.44ms	61.79%	11.09ms	16.44ms	16.79ms	20.56ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/	0	132468	18.07MB	16.56k	3.52k	25.88k	71.25%	7.21ms	15.87ms	217.47ms	97.18%	4.41ms	9.93ms	15.06ms	73.89ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user	0	117695	16.05MB	14.72k	3.66k	24.92k	70.00%	8.03ms	16.18ms	222.90ms	96.22%	4.59ms	11.70ms	17.55ms	65.18ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user/0	0	109060	17.99MB	13.67k	2.09k	18.69k	67.50%	8.41ms	15.83ms	226.37ms	96.87%	6.52ms	11.77ms	16.64ms	51.92ms
bin/server_elixir_plug	http://127.0.0.1:3000/	0	139498	19.02MB	17.46k	2.75k	24.65k	68.75%	6.09ms	13.00ms	205.68ms	98.30%	4.75ms	8.43ms	11.80ms	28.43ms
bin/server_elixir_plug	http://127.0.0.1:3000/user	0	128609	17.54MB	16.09k	3.71k	23.93k	68.75%	7.62ms	15.80ms	215.78ms	97.33%	5.15ms	11.00ms	15.69ms	67.00ms
bin/server_elixir_plug	http://127.0.0.1:3000/user/0	0	135539	22.36MB	16.95k	3.97k	27.29k	66.25%	6.39ms	13.15ms	213.61ms	98.15%	5.12ms	8.63ms	12.28ms	27.72ms
bin/server_go_echo	http://127.0.0.1:3000/	0	276253	30.56MB	34.55k	7.31k	52.76k	65.00%	2.66ms	5.29ms	202.97ms	98.83%	1.86ms	3.47ms	4.92ms	8.16ms
bin/server_go_echo	http://127.0.0.1:3000/user	0	273532	30.26MB	34.24k	8.63k	53.52k	71.25%	3.57ms	11.94ms	206.24ms	98.60%	1.67ms	3.51ms	5.53ms	20.50ms
bin/server_go_echo	http://127.0.0.1:3000/user/0	0	269404	37.51MB	33.70k	6.40k	55.39k	72.50%	3.12ms	8.32ms	207.78ms	99.04%	1.96ms	3.72ms	5.29ms	11.19ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/	0	610519	54.15MB	76.48k	22.98k	120.83k	65.00%	1.27ms	2.88ms	199.23ms	97.36%	0.90ms	1.25ms	2.26ms	5.61ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user	0	620908	55.07MB	78.01k	32.65k	131.95k	60.00%	1.41ms	7.77ms	200.06ms	99.28%	775.00us	0.96ms	1.39ms	7.11ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user/0	0	535732	83.79MB	66.91k	17.61k	102.53k	67.50%	1.69ms	5.08ms	205.96ms	98.43%	1.05ms	1.60ms	3.09ms	7.83ms
bin/server_go_gin	http://127.0.0.1:3000/	0	232969	25.77MB	29.12k	6.68k	46.24k	65.00%	3.28ms	5.11ms	206.07ms	95.29%	2.25ms	4.26ms	6.43ms	15.13ms
bin/server_go_gin	http://127.0.0.1:3000/user	0	233999	25.89MB	29.27k	8.78k	47.15k	73.75%	3.97ms	11.46ms	212.65ms	98.04%	2.10ms	4.15ms	6.67ms	23.13ms
bin/server_go_gin	http://127.0.0.1:3000/user/0	0	227011	31.61MB	28.39k	5.16k	44.18k	65.00%	3.19ms	4.01ms	201.29ms	92.37%	2.22ms	4.24ms	6.46ms	15.18ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/	0	269325	29.79MB	33.72k	4.56k	49.62k	68.75%	2.80ms	6.45ms	205.24ms	99.09%	2.01ms	3.43ms	4.83ms	8.99ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user	0	259825	28.74MB	32.55k	6.58k	62.53k	76.25%	3.19ms	7.08ms	202.57ms	96.86%	1.91ms	3.79ms	5.99ms	16.75ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user/0	0	237488	33.07MB	29.68k	4.92k	42.66k	65.00%	3.61ms	9.33ms	216.84ms	98.56%	2.23ms	4.12ms	6.14ms	15.91ms
bin/server_go_iris	http://127.0.0.1:3000/	0	328609	36.35MB	41.14k	9.14k	63.35k	66.25%	2.71ms	9.19ms	205.97ms	99.02%	1.47ms	2.59ms	4.51ms	11.70ms
bin/server_go_iris	http://127.0.0.1:3000/user	0	313483	34.68MB	39.30k	8.15k	59.51k	71.25%	2.87ms	8.35ms	200.31ms	98.73%	1.65ms	3.18ms	4.98ms	12.45ms
bin/server_go_iris	http://127.0.0.1:3000/user/0	0	313635	43.67MB	39.17k	6.04k	57.48k	68.75%	2.77ms	8.51ms	201.91ms	99.09%	1.65ms	2.83ms	4.68ms	10.62ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/	0	103164	15.94MB	12.89k	4.68k	24.93k	66.25%	5.87ms	6.31ms	216.98ms	95.23%	4.32ms	7.38ms	7.93ms	22.77ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/user	0	104645	16.17MB	13.10k	6.84k	30.59k	61.25%	5.78ms	8.31ms	224.75ms	96.59%	4.09ms	7.27ms	7.87ms	20.81ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/user/0	0	97393	17.83MB	12.15k	3.75k	21.26k	68.75%	7.94ms	11.30ms	236.39ms	96.54%	6.49ms	7.81ms	11.71ms	40.82ms
bin/server_node_express	http://127.0.0.1:3000/	0	11702	1.81MB	1.47k	303.69	2.10k	68.75%	47.77ms	6.85ms	304.39ms	96.28%	47.03ms	50.00ms	51.41ms	55.01ms
bin/server_node_express	http://127.0.0.1:3000/user	0	11385	1.76MB	1.43k	303.55	2.26k	71.25%	62.53ms	12.81ms	329.94ms	95.99%	64.71ms	65.03ms	65.59ms	67.29ms
bin/server_node_express	http://127.0.0.1:3000/user/0	0	10600	1.94MB	1.33k	369.52	2.12k	68.75%	54.90ms	16.82ms	334.96ms	96.93%	54.95ms	55.23ms	63.02ms	70.22ms
bin/server_python_flask	http://127.0.0.1:3000/	0	2573	386.96KB	368.79	212.21	730.00	61.43%	42.92ms	10.71ms	249.61ms	96.93%	42.55ms	44.44ms	45.38ms	52.03ms
bin/server_python_flask	http://127.0.0.1:3000/user	0	2572	386.80KB	444.98	365.57	1.09k	46.55%	46.36ms	17.70ms	844.46ms	98.21%	46.27ms	46.52ms	46.92ms	55.44ms
bin/server_python_flask	http://127.0.0.1:3000/user/0	0	2325	417.77KB	519.40	357.38	1.05k	60.00%	45.96ms	7.87ms	249.99ms	96.13%	46.72ms	46.90ms	47.05ms	47.97ms
bin/server_python_sanic	http://127.0.0.1:3000/	0	219112	24.87MB	27.35k	8.36k	48.48k	66.25%	4.68ms	3.42ms	31.65ms	81.76%	3.76ms	6.41ms	7.84ms	16.94ms
bin/server_python_sanic	http://127.0.0.1:3000/user	0	203034	23.04MB	25.30k	6.40k	50.82k	76.25%	4.85ms	3.21ms	28.10ms	76.85%	4.11ms	7.03ms	7.83ms	16.00ms
bin/server_python_sanic	http://127.0.0.1:3000/user/0	0	209172	29.72MB	26.05k	7.22k	46.38k	75.00%	4.76ms	3.39ms	31.57ms	75.80%	4.00ms	7.02ms	7.82ms	16.56ms
bin/server_rust_iron	http://127.0.0.1:3000/	0	377451	27.00MB	63.28k	47.93k	140.57k	43.33%	269.36us	271.65us	13.58ms	95.91%	225.00us	337.00us	423.00us	818.00us
bin/server_rust_iron	http://127.0.0.1:3000/user	0	357691	25.58MB	59.95k	37.68k	116.23k	48.33%	337.42us	254.98us	19.64ms	91.24%	333.00us	414.00us	494.00us	0.89ms
bin/server_rust_iron	http://127.0.0.1:3000/user/0	0	323302	40.39MB	46.41k	31.45k	103.48k	54.29%	392.30us	810.67us	58.81ms	99.39%	360.00us	487.00us	607.00us	1.03ms
bin/server_rust_nickel	http://127.0.0.1:3000/	0	307276	38.10MB	154.64k	8.27k	161.22k	85.00%	46.75us	24.95us	1.61ms	75.98%	36.00us	50.00us	88.00us	95.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user	0	382918	47.47MB	175.16k	20.53k	208.29k	72.73%	43.38us	29.95us	9.56ms	89.64%	31.00us	64.00us	74.00us	86.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user/0	0	327277	51.50MB	164.46k	12.65k	185.00k	60.00%	49.29us	18.09us	4.00ms	82.48%	47.00us	56.00us	68.00us	84.00us
bin/server_rust_rocket	http://127.0.0.1:3000/	0	341102	29.60MB	171.48k	19.97k	195.75k	60.00%	62.07us	40.21us	1.21ms	77.88%	53.00us	71.00us	130.00us	142.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user	0	332676	28.87MB	111.62k	60.03k	171.57k	66.67%	73.98us	50.96us	10.61ms	93.73%	61.00us	100.00us	118.00us	147.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user/0	0	320420	49.50MB	107.43k	55.10k	166.95k	63.33%	76.82us	49.28us	10.62ms	92.71%	63.00us	106.00us	122.00us	145.00us

[OvermindDL1]

That looks so much better on github than in the terminal... ^.^;

Good to see no error happened on any of them!

Some are clearly faster than others, but most are quite close together. It is sorted by name at this point so ignore the sorting (what should I sort by?).

It is also quite interesting at some of the speed differences between the 3 different routes for a single given server.

In general it looks like from a quick look on that, that python's flask is the overall slowest (by far!), with node's express not far behind. Go seemed to have the most impressive showing (I really really dislike the 'language' itself though, blegh...). Rust was not far behind Go. Elixir was a good middle ground. Crystal was pretty far behind though, I guess it's multi-threading is not up to par quite yet unlike many of the other frameworks (I am running the latest/nightly versions of crystal/rust/go/python3, though my elixir is one version out of date now since a new version just came out).

Now I think I may let it run with a higher connections count with 5s warmup and 30s runtime per test.

[OvermindDL1]

First I decided to test 'just' rust with 10k concurrent connections, whoops, I need to increase one of my OS limits, glad I did this test first... >.>

On the plus side for as many threads as it was able to create, (EDIT: misread, amended this sentance ^.^) Rust did quite well, it did not really scale any better with the connection count though.

The results:

Total Cores: 16
Concurrent Connections: 10000

Processing servers:

• bin/server_rust_iron
• bin/server_rust_nickel
• bin/server_rust_rocket

Processing: bin/server_rust_iron

• Launching server: bin/server_rust_iron
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/98700298838429703802256171858
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/85167260135869599983225488542
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_rust_iron

Processing: bin/server_rust_nickel

• Launching server: bin/server_rust_nickel

• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/52812330939115869124678431264
• • • Warming up URL: http://127.0.0.1:3000/user

• URL: http://127.0.0.1:3000/

• • Performing test for 30s...
• • Test Complete

• URL: http://127.0.0.1:3000/user/78956953426456587050998785165

• • Performing test for 30s...
• • Test Complete

• URL: http://127.0.0.1:3000/user

• • Performing test for 30s...
    ../post.lua: cannot open ../post.lua: Too many open files
    unable to create thread 7: Too many open files
• • • Test returned an unsuccessful error code, output of test:
• • Test Complete

• Shutting down server: bin/server_rust_nickel

Processing: bin/server_rust_rocket

• Launching server: bin/server_rust_rocket
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/59184218110503514755998552113
• • • Warming up URL: http://127.0.0.1:3000/user
      ../post.lua: cannot open ../post.lua: Too many open files
      unable to create thread 7: Too many open files
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/77572417918735240821734743966
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
    ../post.lua: cannot open ../post.lua: Too many open files
• • Test Complete
• Shutting down server: bin/server_rust_rocket

Path	URL	Errors	Total Requests Count	Total Requests Throughput	Req/s Avg	Req/s Stdev	Req/s Max	Req/s +/-	Latency Avg	Latency Stdev	Latency Max	Latency +/-	50%	75%	90%	99%
bin/server_rust_iron	http://127.0.0.1:3000/	0	10947034	782.99MB	91.68k	49.63k	140.01k	75.00%	229.33us	174.20us	19.35ms	90.29%	225.00us	273.00us	368.00us	621.00us
bin/server_rust_iron	http://127.0.0.1:3000/user	0	10518663	752.35MB	88.12k	47.81k	143.83k	72.25%	251.87us	211.54us	21.36ms	92.88%	234.00us	286.00us	419.00us	724.00us
bin/server_rust_iron	http://127.0.0.1:3000/user/0	0	9505634	1.16GB	106.14k	23.18k	132.44k	66.44%	282.64us	241.00us	25.26ms	95.02%	250.00us	320.00us	477.00us	846.00us
bin/server_rust_nickel	http://127.0.0.1:3000/	0	4765504	590.82MB	159.67k	7.48k	176.46k	67.00%	78.56us	30.30us	4.63ms	60.60%	61.00us	114.00us	122.00us	131.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user/0	0	5064768	796.97MB	169.62k	11.48k	221.65k	89.00%	74.91us	29.87us	7.85ms	61.07%	57.00us	109.00us	116.00us	123.00us
bin/server_rust_rocket	http://127.0.0.1:3000/	0	8530304	740.30MB	142.95k	17.75k	195.35k	70.50%	73.29us	51.06us	12.10ms	81.93%	66.00us	86.00us	161.00us	174.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user	0	7442954	645.93MB	124.69k	42.14k	199.62k	67.00%	81.46us	53.90us	12.01ms	71.86%	76.00us	91.00us	171.00us	184.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user/0	0	4762954	735.85MB	159.53k	2.34k	164.68k	66.67%	116.52us	45.51us	14.60ms	75.55%	98.00us	114.00us	192.00us	202.00us

[OvermindDL1]

With my system limit enhanced, I decided to try elixir with 50K connections (I set the file descriptor limit to 65536) and warmup of 3s and runtime of 10s, and the results:

Total Cores: 16
Concurrent Connections: 50000

Processing servers:

• bin/server_elixir_phoenix
• bin/server_elixir_plug

Processing: bin/server_elixir_phoenix

• Launching server: bin/server_elixir_phoenix
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/78859239144763392299227572306
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 10s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/18603197934598969687347993941
• • Performing test for 10s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 10s...
• • Test Complete
• Shutting down server: bin/server_elixir_phoenix

Processing: bin/server_elixir_plug

• Launching server: bin/server_elixir_plug
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/48746203591989270871020485155
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 10s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/29871532825000663200765424721
• • Performing test for 10s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 10s...
• • Test Complete
• Shutting down server: bin/server_elixir_plug

Path	URL	Errors	Total Requests Count	Total Requests Throughput	Req/s Avg	Req/s Stdev	Req/s Max	Req/s +/-	Latency Avg	Latency Stdev	Latency Max	Latency +/-	50%	75%	90%	99%
bin/server_elixir_phoenix	http://127.0.0.1:3000/	0	6081	849.20KB	7.43k	5.58k	20.56k	87.50%	21.66ms	11.24ms	59.65ms	66.06%	19.39ms	27.52ms	37.47ms	55.00ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user	0	3736	521.73KB	4.62k	3.85k	13.86k	87.50%	22.12ms	10.99ms	65.17ms	68.31%	19.83ms	27.85ms	37.33ms	54.95ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user/0	0	8485	1.40MB	10.38k	8.12k	25.88k	75.00%	18.82ms	11.21ms	63.60ms	72.47%	16.80ms	22.51ms	32.13ms	60.33ms
bin/server_elixir_plug	http://127.0.0.1:3000/	0	6990	0.95MB	8.69k	8.42k	27.85k	87.50%	19.81ms	12.04ms	60.48ms	65.64%	16.72ms	27.13ms	38.28ms	53.77ms
bin/server_elixir_plug	http://127.0.0.1:3000/user	0	5982	835.38KB	7.28k	8.06k	25.71k	87.50%	20.12ms	11.42ms	62.21ms	66.67%	18.91ms	26.26ms	35.47ms	53.07ms
bin/server_elixir_plug	http://127.0.0.1:3000/user/0	0	5525	0.91MB	6.72k	2.67k	11.52k	75.00%	16.26ms	8.07ms	59.00ms	74.14%	14.98ms	20.40ms	25.67ms	48.97ms

[OvermindDL1]

50k is definitely a bit more than my system can handle it seems, I think 1k is a good default value then, after all even rust didn't scale to higher concurrent connections later but it did seem to sustain it well. Now it is time for the 5s warmup, 30s runtime tests with 1k connections. I went ahead and pasted in the shell command I used to launch it as well so all settings can be seen, it defaults to 1000 concurrent connections and it is limited to half the total core count of sending the surge of requests, 'most' servers absolutely cap out every CPU at 100%, though some are obviously single threaded, like python and node. Memory usage did not seem to flucuate much among any of the servers though, baseline of system was about 390megs in use, rust seemed to spike to between 420-600megs (the higher memory usage one was the faster one as expected, I should have it report cpu/memory usage of the system at baseline and during tests... maybe later), crystal seemed to average ~500megs, though it seemed to only be using 1 CPU total (~102% CPU usage instead of the expected ~1200%), shouldn't crystal be heavily multi-threaded, this would definitely explain its poor ranking, though perhaps it's trying to minimize context switching cost by passing information between threads (going multi-threaded will definitely add in a hit on speed, just a question of if the multi-threading itself will make up for it or not, for note, the siegeing engine is using ~10% of each core), and wow when the go servers start running all cores are pegged to 100% again, with surprisingly little memory usage, only around ~438M system usage (why does the language syntax suck so much, blegh...), all the 4(5?) other go servers used almost identical memory too, fasthttprouter was just a 'tiny' bit less memory than the others (~432M) and iris using just a tiny bit more (~450), and pythons flask ate about ~418m and only 1 core, however python's sanic hit near 100% CPU usage(!) and ate ~520M,, it is multi-process as I saw many processes startup, then elixir phoenix started, ate about 80% of each CPU (it's VM tries to be CPU friendly as much as possible) and ate 550M, elixir plug is identical, which makes sense since phoenix just a full web framework on top of plug, and nodes clusterexpress is obviously mulit-core, it hits 100% of all CPU's, however it eats an amazing 1.24G of RAM (!?), express by itself barely hits 100% of one CPU, but only eats ~477M (which is still high compared to most servers here), and the output:

╰─➤ ../stats.exs -c 1000 -w 5 -d 30 rust crystal go flask sanic elixir node
Total Cores: 16
Concurrent Connections: 1000

Processing servers:

• bin/server_rust_iron
• bin/server_rust_nickel
• bin/server_rust_rocket
• bin/server_crystal_kemal
• bin/server_crystal_raze
• bin/server_crystal_router_cr
• bin/server_go_echo
• bin/server_go_fasthttprouter
• bin/server_go_gin
• bin/server_go_gorilla_mux
• bin/server_go_iris
• bin/server_python_flask
• bin/server_python_sanic
• bin/server_elixir_phoenix
• bin/server_elixir_plug
• bin/server_node_clusterexpress
• bin/server_node_express

Processing: bin/server_rust_iron

• Launching server: bin/server_rust_iron
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/80310875046755178217836306950
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/52328741465164477178078433374
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_rust_iron

Processing: bin/server_rust_nickel

• Launching server: bin/server_rust_nickel
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/23135401432731901026974054297
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/73203904315018441280360650672
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_rust_nickel

Processing: bin/server_rust_rocket

• Launching server: bin/server_rust_rocket
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/43581030434907989730338000309
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/85725910617373397745696086163
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_rust_rocket

Processing: bin/server_crystal_kemal

• Launching server: bin/server_crystal_kemal
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/72417623314037919859174327274
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/24670413940687467172971259775
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_crystal_kemal

Processing: bin/server_crystal_raze

• Launching server: bin/server_crystal_raze
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/86989114112146860722621803896
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/31559412054872301359254445186
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_crystal_raze

Processing: bin/server_crystal_router_cr

• Launching server: bin/server_crystal_router_cr
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/61325979598113381349064452394
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/88373378253149565668746247018
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_crystal_router_cr

Processing: bin/server_go_echo

• Launching server: bin/server_go_echo
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/65665667639758795677388889207
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/11117792787524171124711842293
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_go_echo

Processing: bin/server_go_fasthttprouter

• Launching server: bin/server_go_fasthttprouter
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/5094758044760646999313060026
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/83714651432790889256685453938
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_go_fasthttprouter

Processing: bin/server_go_gin

• Launching server: bin/server_go_gin
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/29325495156784872676357272482
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/81305628909170656013696154156
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_go_gin

Processing: bin/server_go_gorilla_mux

• Launching server: bin/server_go_gorilla_mux
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/59573285147826025337208916933
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/82142077731965901694904613708
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_go_gorilla_mux

Processing: bin/server_go_iris

• Launching server: bin/server_go_iris
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/47522884156756950638085683497
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/12327963608096765486640397351
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_go_iris

Processing: bin/server_python_flask

• Launching server: bin/server_python_flask
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/48648282362743669546459872665
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/37242558110150762339099306876
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_python_flask

Processing: bin/server_python_sanic

• Launching server: bin/server_python_sanic
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/62426265767722551878239831725
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/67376208805439954190347051178
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_python_sanic

Processing: bin/server_elixir_phoenix

• Launching server: bin/server_elixir_phoenix
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/19573053844363156278888046150
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/77373582634948303043906905896
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_elixir_phoenix

Processing: bin/server_elixir_plug

• Launching server: bin/server_elixir_plug
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/43529385107880424446196911768
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/89123228724446833323505187472
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_elixir_plug

Processing: bin/server_node_clusterexpress

• Launching server: bin/server_node_clusterexpress
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/88699828112656712698407342555
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/87711301316659405169695176803
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_node_clusterexpress

Processing: bin/server_node_express

• Launching server: bin/server_node_express
• • Warming up for 5s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/83956776074189458077863548676
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/22600418458423928970803820565
• • Performing test for 30s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 30s...
• • Test Complete
• Shutting down server: bin/server_node_express

Path	URL	Errors	Total Requests Count	Total Requests/s	Total Requests Throughput	Total Throughput/s	Req/s Avg	Req/s Stdev	Req/s Max	Req/s +/-	Latency Avg	Latency Stdev	Latency Max	Latency +/-	50%	75%	90%	99%
bin/server_crystal_kemal	http://127.0.0.1:3000/	0	2018784	207.93MB	67159.57	6.92MB	8.45k	562.28	9.99k	69.96%	14.86ms	3.51ms	31.48ms	77.44%	13.03ms	13.88ms	20.72ms	25.66ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user	0	1603828	165.19MB	53390.93	5.50MB	6.71k	431.86	7.59k	66.08%	18.68ms	4.71ms	231.04ms	63.76%	15.25ms	23.75ms	24.21ms	29.72ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user/0	0	1413974	186.09MB	47033.96	6.19MB	5.92k	331.30	7.20k	60.75%	21.21ms	4.60ms	39.13ms	68.33%	17.64ms	25.62ms	26.06ms	31.61ms
bin/server_crystal_raze	http://127.0.0.1:3000/	0	2079720	122.97MB	69166.99	4.09MB	8.70k	358.70	10.01k	80.12%	14.37ms	3.28ms	228.49ms	58.93%	16.70ms	17.20ms	17.47ms	18.02ms
bin/server_crystal_raze	http://127.0.0.1:3000/user	0	1774370	104.91MB	59004.83	3.49MB	7.42k	484.76	8.84k	73.17%	16.87ms	3.12ms	217.93ms	73.40%	18.02ms	18.36ms	18.73ms	24.09ms
bin/server_crystal_raze	http://127.0.0.1:3000/user/0	0	1648147	144.61MB	54836.04	4.81MB	6.90k	450.90	8.43k	68.79%	18.13ms	3.83ms	233.54ms	79.04%	19.22ms	19.45ms	19.68ms	22.40ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/	0	2402211	142.04MB	79923.18	4.73MB	10.05k	439.44	11.36k	73.04%	12.47ms	2.97ms	219.21ms	66.85%	10.53ms	16.41ms	16.72ms	17.09ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user	0	2019711	119.42MB	67223.57	3.97MB	8.45k	499.56	10.01k	76.88%	14.81ms	3.52ms	221.83ms	72.25%	12.76ms	18.10ms	18.45ms	21.74ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user/0	0	2111187	185.23MB	70214.92	6.16MB	8.84k	578.54	10.12k	69.83%	14.21ms	3.41ms	31.58ms	70.74%	11.94ms	17.75ms	18.28ms	23.70ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/	0	3903425	532.33MB	129848.18	17.71MB	16.35k	2.67k	25.76k	67.52%	8.54ms	8.56ms	236.30ms	87.20%	8.65ms	13.41ms	18.41ms	28.89ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user	0	3441921	469.39MB	114473.32	15.61MB	14.41k	2.70k	25.33k	68.44%	9.78ms	9.67ms	271.64ms	56.82%	9.96ms	15.61ms	21.38ms	33.62ms
bin/server_elixir_phoenix	http://127.0.0.1:3000/user/0	0	3399469	560.86MB	113011.72	18.65MB	14.23k	2.05k	33.66k	70.53%	9.48ms	8.78ms	221.57ms	57.13%	9.87ms	14.66ms	19.66ms	30.02ms
bin/server_elixir_plug	http://127.0.0.1:3000/	0	4204063	573.33MB	139973.24	19.09MB	17.62k	2.28k	26.51k	68.81%	7.69ms	7.53ms	230.39ms	59.24%	7.92ms	11.75ms	15.85ms	24.80ms
bin/server_elixir_plug	http://127.0.0.1:3000/user	0	3844247	524.26MB	127905.48	17.44MB	16.09k	3.13k	33.43k	70.28%	8.96ms	11.49ms	435.65ms	91.96%	8.79ms	13.97ms	19.41ms	30.83ms
bin/server_elixir_plug	http://127.0.0.1:3000/user/0	0	3953662	652.30MB	131530.33	21.70MB	16.56k	2.09k	30.18k	68.27%	8.11ms	7.61ms	222.75ms	59.44%	8.37ms	12.29ms	16.51ms	25.51ms
bin/server_go_echo	http://127.0.0.1:3000/	0	8143056	0.88GB	270536.69	29.93MB	34.09k	4.97k	86.67k	72.81%	3.60ms	4.58ms	220.04ms	96.15%	2.85ms	4.68ms	6.54ms	11.15ms
bin/server_go_echo	http://127.0.0.1:3000/user	0	8083816	0.87GB	268705.33	29.73MB	33.80k	4.72k	54.25k	69.76%	3.96ms	4.73ms	211.71ms	93.15%	2.86ms	4.89ms	7.45ms	19.77ms
bin/server_go_echo	http://127.0.0.1:3000/user/0	0	7878681	1.07GB	261750.2	36.45MB	32.96k	4.68k	63.40k	72.16%	3.83ms	7.59ms	418.43ms	98.87%	2.95ms	4.82ms	6.75ms	11.81ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/	0	16616387	1.44GB	552545.67	49.01MB	69.55k	17.93k	133.31k	63.88%	1.80ms	3.09ms	203.59ms	96.16%	1.27ms	2.03ms	3.41ms	7.14ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user	0	15773986	1.37GB	524262.17	46.50MB	66.01k	18.00k	142.84k	66.85%	1.88ms	2.81ms	203.84ms	95.02%	1.34ms	2.18ms	3.58ms	7.30ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user/0	0	14871569	2.27GB	494503.02	77.34MB	62.26k	15.53k	106.90k	65.20%	1.96ms	1.98ms	203.03ms	90.81%	1.40ms	2.39ms	3.80ms	7.44ms
bin/server_go_gin	http://127.0.0.1:3000/	0	6905333	763.91MB	229436.3	25.38MB	28.90k	4.02k	82.67k	73.60%	4.51ms	4.83ms	209.18ms	93.31%	3.57ms	5.71ms	8.10ms	20.11ms
bin/server_go_gin	http://127.0.0.1:3000/user	0	6899553	763.27MB	229241.65	25.36MB	28.87k	4.46k	62.87k	70.94%	5.10ms	5.89ms	227.78ms	90.95%	3.52ms	6.04ms	10.33ms	28.18ms
bin/server_go_gin	http://127.0.0.1:3000/user/0	0	6740060	0.92GB	223956.08	31.18MB	28.20k	4.01k	79.30k	74.32%	4.61ms	4.29ms	206.48ms	88.73%	3.66ms	5.86ms	8.35ms	20.68ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/	0	7894014	0.85GB	262794.54	29.07MB	33.07k	4.11k	52.68k	68.96%	3.71ms	2.80ms	205.95ms	82.18%	2.98ms	4.84ms	6.85ms	13.17ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user	0	7761618	858.64MB	257894.84	28.53MB	32.46k	4.76k	67.66k	71.92%	4.20ms	4.18ms	210.52ms	90.94%	3.04ms	5.17ms	7.99ms	22.16ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user/0	0	6974864	0.95GB	231744.78	32.27MB	29.21k	4.15k	106.12k	75.06%	4.35ms	4.63ms	221.90ms	93.58%	3.47ms	5.60ms	7.82ms	16.50ms
bin/server_go_iris	http://127.0.0.1:3000/	0	9526387	1.03GB	316501.62	35.01MB	39.87k	7.19k	90.95k	70.80%	3.03ms	2.32ms	200.76ms	81.96%	2.38ms	3.95ms	5.91ms	10.68ms
bin/server_go_iris	http://127.0.0.1:3000/user	0	9348087	1.01GB	310593.53	34.36MB	39.11k	7.05k	81.36k	70.07%	3.31ms	3.74ms	216.04ms	92.12%	2.38ms	4.18ms	6.43ms	14.32ms
bin/server_go_iris	http://127.0.0.1:3000/user/0	0	9063856	1.23GB	301615.13	42.00MB	37.96k	6.91k	70.77k	70.60%	3.21ms	2.91ms	202.41ms	88.19%	2.51ms	4.14ms	6.22ms	11.41ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/	0	3227217	498.59MB	107217.04	16.56MB	13.51k	1.49k	33.96k	77.45%	9.12ms	4.83ms	224.02ms	77.01%	7.74ms	11.37ms	15.41ms	23.35ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/user	0	3202508	494.77MB	106414.09	16.44MB	13.40k	1.38k	56.02k	84.91%	9.14ms	4.27ms	152.68ms	73.72%	7.74ms	11.51ms	15.44ms	20.43ms
bin/server_node_clusterexpress	http://127.0.0.1:3000/user/0	0	3063205	560.89MB	101878.67	18.65MB	12.82k	1.63k	18.45k	63.34%	9.72ms	5.06ms	219.98ms	78.66%	7.90ms	11.60ms	15.71ms	24.49ms
bin/server_node_express	http://127.0.0.1:3000/	0	348348	53.82MB	11588.36	1.79MB	1.46k	160.56	2.27k	68.38%	85.64ms	18.43ms	683.13ms	95.63%	86.37ms	86.65ms	87.29ms	89.20ms
bin/server_node_express	http://127.0.0.1:3000/user	0	344033	53.15MB	11449.4	1.77MB	1.44k	141.12	2.18k	72.50%	86.68ms	36.28ms	1.08s	95.66%	87.24ms	87.42ms	87.58ms	89.13ms
bin/server_node_express	http://127.0.0.1:3000/user/0	0	307982	56.39MB	10244.34	1.88MB	1.30k	353.95	5.29k	89.12%	93.11ms	45.15ms	1.09s	89.37%	97.27ms	98.06ms	106.02ms	130.82ms
bin/server_python_flask	http://127.0.0.1:3000/	179	14112	2.07MB	469.67	70.63KB	438.22	435.07	2.97k	77.17%	49.68ms	89.53ms	3.28s	96.03%	44.57ms	44.93ms	45.39ms	601.73ms
bin/server_python_flask	http://127.0.0.1:3000/user	272	20373	2.99MB	678.66	102.06KB	338.75	352.68	1.19k	76.98%	72.15ms	151.94ms	3.94s	96.12%	46.27ms	46.57ms	47.06ms	612.41ms
bin/server_python_flask	http://127.0.0.1:3000/user/0	0	4762	855.67KB	158.26	28.44KB	202.87	186.67	780.00	70.23%	71.33ms	120.57ms	878.83ms	91.35%	37.02ms	62.84ms	140.42ms	832.97ms
bin/server_python_sanic	http://127.0.0.1:3000/	0	6507225	738.49MB	216248.64	24.54MB	27.26k	6.10k	59.77k	69.27%	4.60ms	3.34ms	51.46ms	77.70%	3.86ms	6.63ms	7.71ms	16.55ms
bin/server_python_sanic	http://127.0.0.1:3000/user	0	6027117	684.00MB	200373.66	22.74MB	25.26k	5.08k	55.16k	69.92%	4.93ms	3.38ms	59.31ms	77.22%	4.20ms	7.05ms	7.83ms	16.84ms
bin/server_python_sanic	http://127.0.0.1:3000/user/0	0	6307807	0.88GB	209760.13	29.81MB	26.43k	6.08k	57.14k	68.32%	4.71ms	3.30ms	48.53ms	77.23%	3.97ms	6.90ms	7.73ms	16.39ms
bin/server_rust_iron	http://127.0.0.1:3000/	0	11444364	818.56MB	380822.91	27.24MB	54.76k	38.07k	135.15k	57.05%	309.11us	303.98us	46.98ms	96.89%	301.00us	389.00us	477.00us	842.00us
bin/server_rust_iron	http://127.0.0.1:3000/user	0	10817548	773.73MB	360281.54	25.77MB	60.41k	35.89k	136.57k	66.72%	313.84us	261.07us	30.24ms	95.64%	311.00us	392.00us	477.00us	751.00us
bin/server_rust_iron	http://127.0.0.1:3000/user/0	0	9772517	1.19GB	324950.53	40.60MB	40.92k	34.26k	132.12k	67.62%	346.60us	289.12us	31.46ms	91.79%	330.00us	453.00us	576.00us	0.99ms
bin/server_rust_nickel	http://127.0.0.1:3000/	0	4870844	603.88MB	161986.71	20.08MB	163.19k	12.98k	205.17k	76.67%	76.42us	32.13us	12.96ms	66.96%	60.00us	112.00us	121.00us	130.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user	0	9037151	1.09GB	300775.0	37.29MB	151.41k	13.86k	210.60k	69.83%	51.94us	23.02us	10.21ms	83.10%	46.00us	68.00us	77.00us	87.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user/0	0	10066517	1.55GB	334773.22	52.68MB	112.43k	58.13k	189.35k	54.00%	50.53us	22.25us	10.98ms	88.43%	51.00us	58.00us	67.00us	85.00us
bin/server_rust_rocket	http://127.0.0.1:3000/	0	10908846	0.92GB	362640.69	31.47MB	121.84k	50.66k	204.76k	54.89%	68.20us	42.09us	11.98ms	94.41%	64.00us	88.00us	102.00us	129.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user	0	10859171	0.92GB	361154.03	31.34MB	121.28k	32.99k	187.25k	58.33%	72.60us	44.34us	11.86ms	95.20%	71.00us	90.00us	105.00us	141.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user/0	0	9873379	1.49GB	328320.21	50.72MB	110.28k	61.67k	194.74k	57.22%	75.67us	46.89us	12.18ms	93.70%	68.00us	101.00us	115.00us	146.00us

[OvermindDL1]

It seems to be about the same relative values as compared to the 1s test (^.^:), go just stomps everything else with rust close behind, etc... etc...

Interestingly flask got overloaded well enough that it started having a tiny bit of errors overall, still poor compared to everything else that is error-less. ^.^

Most things seem to have a pretty good worst-case latency as well (Rust had the best worst-case by far of anything else). To continue tests I think I'd need to start overwhelming them and seeing when they start to fail.

Should I toss my script into the repo? Not pretty at all but it works.

[OvermindDL1]

Also apparently I had req/s and throughput backwards, ignore that, it's fixed in the code now. :-)

[kalicki]

I think we should monitor memory usage, that should have a high value in the analysis. I missed analyzing Lucky (Crystal), I didn't find it in your analysis.

[kalicki]

I think we should improve the presentation of the results and even create a kind of graphics in this style, we can even put the results in different environments like win, Linux, Mac and their configs

eg.
https://screenshots.firefox.com/6yABrQ90SYXmC0j8/markojs.com => https://markojs.com/

[OvermindDL1]

Yeah this is a very raw dump of data, it could easily be output in any way.

And Lucky must not have been in the latest git master as that is what I used. ^.^

[OvermindDL1]

I whipped up a C++ server using libevent (libevent is the library that a lot, like a LOT of servers use, such as most servers in node, go, etc...) using the http(s) server API (this is a full server routing framework and all), I figure it should be a good baseline as most things build on top of libevent (should I PR it in?):

Results:

╰─➤ ../stats.exs -w 1 -d 2 evhtp rust crystal go
Total Cores: 16
Concurrent Connections: 1000

Processing servers:

• bin/server_cpp_evhtp
• bin/server_rust_iron
• bin/server_rust_nickel
• bin/server_rust_rocket
• bin/server_crystal_kemal
• bin/server_crystal_raze
• bin/server_crystal_router_cr
• bin/server_go_echo
• bin/server_go_fasthttprouter
• bin/server_go_gin
• bin/server_go_gorilla_mux
• bin/server_go_iris

Processing: bin/server_cpp_evhtp

• Launching server: bin/server_cpp_evhtp
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/85052466891014107342201867974
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/63776479130153606972920926661
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_cpp_evhtp

Processing: bin/server_rust_iron

• Launching server: bin/server_rust_iron
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/50960473321844660981813177558
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/14935842623295547926488962464
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_rust_iron

Processing: bin/server_rust_nickel

• Launching server: bin/server_rust_nickel
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/41094751679689971794331836364
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/20138720514572116449183414227
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_rust_nickel

Processing: bin/server_rust_rocket

• Launching server: bin/server_rust_rocket
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/78685640120807708752827453990
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/14786272974517776962963874322
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_rust_rocket

Processing: bin/server_crystal_kemal

• Launching server: bin/server_crystal_kemal
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/54241887579774356660508436422
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/33170668274847421256260100416
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_crystal_kemal

Processing: bin/server_crystal_raze

• Launching server: bin/server_crystal_raze
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/49483462420708822452097699262
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/88847091332806675615694442889
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_crystal_raze

Processing: bin/server_crystal_router_cr

• Launching server: bin/server_crystal_router_cr
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/92880162931154408748190654382
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/65961200211467837485312574020
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_crystal_router_cr

Processing: bin/server_go_echo

• Launching server: bin/server_go_echo
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/47497974667024372070828129764
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/52048888220067301400020550076
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_go_echo

Processing: bin/server_go_fasthttprouter

• Launching server: bin/server_go_fasthttprouter
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/13106215207284706079191356801
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/78140608453618012880399138375
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_go_fasthttprouter

Processing: bin/server_go_gin

• Launching server: bin/server_go_gin
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/3528480840031510853279355648
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/5975003814194492545887032882
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_go_gin

Processing: bin/server_go_gorilla_mux

• Launching server: bin/server_go_gorilla_mux
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/78308744715842431603902706820
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/13179253381389641733706742759
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_go_gorilla_mux

Processing: bin/server_go_iris

• Launching server: bin/server_go_iris
• • Warming up for 1s 3 times...
• • • Warming up URL: http://127.0.0.1:3000/
• • • Warming up URL: http://127.0.0.1:3000/user/40823519989550834248632716959
• • • Warming up URL: http://127.0.0.1:3000/user
• URL: http://127.0.0.1:3000/
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user/43977034345614673322365402250
• • Performing test for 2s...
• • Test Complete
• URL: http://127.0.0.1:3000/user
• • Performing test for 2s...
• • Test Complete
• Shutting down server: bin/server_go_iris

Path	URL	Errors	Total Requests Count	Total Requests/s	Total Requests Throughput	Total Throughput/s	Req/s Avg	Req/s Stdev	Req/s Max	Req/s +/-	Latency Avg	Latency Stdev	Latency Max	Latency +/-	50%	75%	90%	99%
bin/server_cpp_evhtp	http://127.0.0.1:3000/	0	1427890	687945.81	87.15MB	41.99MB	89.24k	25.87k	130.30k	55.62%	1.86ms	2.76ms	27.33ms	87.73%	613.00us	2.01ms	5.38ms	12.91ms
bin/server_cpp_evhtp	http://127.0.0.1:3000/user	0	1353714	662471.43	82.62MB	40.43MB	84.61k	26.36k	127.38k	56.25%	1.83ms	2.47ms	26.10ms	87.14%	645.00us	2.29ms	5.02ms	11.64ms
bin/server_cpp_evhtp	http://127.0.0.1:3000/user/0	0	1344998	646974.87	120.57MB	58.00MB	84.21k	25.52k	124.68k	53.12%	1.88ms	2.59ms	28.04ms	87.52%	663.00us	2.26ms	5.13ms	12.45ms
bin/server_crystal_kemal	http://127.0.0.1:3000/	0	127254	62676.29	13.11MB	6.46MB	8.00k	0.90k	9.97k	80.00%	14.82ms	3.28ms	33.93ms	77.38%	13.66ms	13.98ms	20.92ms	23.29ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user	0	86446	42190.2	8.90MB	4.35MB	5.43k	486.91	6.29k	70.62%	22.84ms	4.45ms	51.90ms	73.43%	24.09ms	24.38ms	25.18ms	36.39ms
bin/server_crystal_kemal	http://127.0.0.1:3000/user/0	0	78361	38510.06	10.31MB	5.07MB	4.92k	267.68	5.78k	70.62%	25.38ms	2.65ms	32.70ms	90.12%	25.96ms	26.12ms	26.29ms	31.22ms
bin/server_crystal_raze	http://127.0.0.1:3000/	0	136821	66842.83	8.09MB	3.95MB	8.59k	593.47	10.10k	80.62%	14.48ms	3.55ms	31.16ms	77.41%	12.15ms	17.69ms	18.36ms	23.95ms
bin/server_crystal_raze	http://127.0.0.1:3000/user	0	135620	66835.34	8.02MB	3.95MB	8.51k	340.56	8.95k	85.62%	14.69ms	3.28ms	26.70ms	71.21%	12.75ms	19.15ms	19.60ms	21.68ms
bin/server_crystal_raze	http://127.0.0.1:3000/user/0	0	124093	60863.38	10.89MB	5.34MB	7.79k	656.10	8.62k	90.62%	16.05ms	3.64ms	40.21ms	72.04%	14.04ms	20.37ms	20.81ms	29.07ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/	0	150720	73293.74	8.91MB	4.33MB	9.46k	1.20k	12.64k	78.12%	12.45ms	7.31ms	218.55ms	96.62%	11.02ms	14.40ms	17.12ms	23.94ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user	0	130501	64578.02	7.72MB	3.82MB	8.19k	424.34	9.27k	58.12%	15.27ms	3.60ms	26.85ms	78.21%	12.91ms	18.74ms	19.36ms	24.39ms
bin/server_crystal_router_cr	http://127.0.0.1:3000/user/0	0	137251	67077.85	12.04MB	5.89MB	8.62k	759.16	10.10k	73.12%	14.20ms	3.25ms	215.99ms	76.03%	12.74ms	15.05ms	19.09ms	22.13ms
bin/server_go_echo	http://127.0.0.1:3000/	0	545981	264975.13	60.40MB	29.31MB	34.09k	4.69k	47.53k	66.25%	4.46ms	14.23ms	210.97ms	98.67%	2.41ms	4.27ms	6.25ms	61.39ms
bin/server_go_echo	http://127.0.0.1:3000/user	0	535549	259563.09	59.25MB	28.71MB	33.45k	4.95k	48.13k	65.00%	3.70ms	7.29ms	208.84ms	96.72%	2.40ms	4.45ms	6.96ms	18.17ms
bin/server_go_echo	http://127.0.0.1:3000/user/0	0	516149	250105.88	71.87MB	34.82MB	32.26k	4.32k	47.08k	68.12%	3.46ms	5.30ms	216.23ms	96.41%	2.57ms	4.61ms	6.61ms	11.96ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/	0	1079061	526945.72	95.70MB	46.74MB	67.62k	15.57k	107.68k	68.12%	1.61ms	1.43ms	21.69ms	88.18%	1.12ms	1.86ms	3.24ms	7.38ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user	0	1026163	503337.64	91.01MB	44.64MB	64.18k	15.05k	109.47k	68.12%	1.80ms	2.80ms	205.80ms	93.66%	1.17ms	2.04ms	3.67ms	8.62ms
bin/server_go_fasthttprouter	http://127.0.0.1:3000/user/0	0	981917	475993.96	153.57MB	74.45MB	61.26k	13.09k	101.01k	68.75%	1.85ms	1.98ms	60.01ms	89.38%	1.16ms	2.10ms	3.98ms	10.18ms
bin/server_go_gin	http://127.0.0.1:3000/	0	453463	221348.85	50.16MB	24.49MB	28.34k	4.19k	41.77k	68.75%	5.55ms	17.95ms	415.27ms	98.39%	3.05ms	5.06ms	7.57ms	79.31ms
bin/server_go_gin	http://127.0.0.1:3000/user	0	449422	220660.12	49.72MB	24.41MB	28.10k	5.22k	44.00k	69.38%	5.21ms	11.55ms	232.69ms	95.91%	3.04ms	5.49ms	9.53ms	30.46ms
bin/server_go_gin	http://127.0.0.1:3000/user/0	0	436164	211764.33	60.31MB	29.28MB	27.26k	3.88k	43.44k	70.00%	4.52ms	9.15ms	212.63ms	97.71%	3.14ms	5.30ms	7.67ms	22.05ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/	0	521280	253238.23	57.67MB	28.01MB	32.59k	5.23k	49.03k	70.00%	3.89ms	9.99ms	230.37ms	98.68%	2.52ms	4.38ms	6.45ms	16.33ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user	0	503400	247116.6	55.69MB	27.34MB	31.43k	5.94k	47.71k	72.50%	4.87ms	13.98ms	217.81ms	98.00%	2.52ms	4.65ms	7.55ms	50.91ms
bin/server_go_gorilla_mux	http://127.0.0.1:3000/user/0	0	459921	222029.71	64.04MB	30.91MB	28.71k	3.78k	40.53k	68.75%	3.77ms	6.19ms	240.05ms	97.34%	2.82ms	4.78ms	6.78ms	13.48ms
bin/server_go_iris	http://127.0.0.1:3000/	0	615823	299482.13	68.13MB	33.13MB	38.49k	6.00k	58.30k	70.00%	3.21ms	7.84ms	210.11ms	98.49%	2.06ms	3.69ms	5.88ms	12.78ms
bin/server_go_iris	http://127.0.0.1:3000/user	0	596330	293099.65	65.97MB	32.42MB	37.26k	6.66k	60.31k	72.50%	3.67ms	9.00ms	208.04ms	97.88%	2.20ms	4.02ms	6.60ms	17.81ms
bin/server_go_iris	http://127.0.0.1:3000/user/0	0	580876	282486.16	80.88MB	39.33MB	36.23k	5.38k	52.11k	70.00%	3.77ms	11.33ms	214.13ms	99.00%	2.18ms	3.93ms	6.00ms	15.13ms
bin/server_rust_iron	http://127.0.0.1:3000/	0	801804	387449.37	57.35MB	27.71MB	57.59k	40.74k	114.62k	44.29%	273.99us	191.92us	8.11ms	89.95%	269.00us	337.00us	412.00us	730.00us
bin/server_rust_iron	http://127.0.0.1:3000/user	0	776765	377428.76	55.56MB	27.00MB	65.10k	32.97k	117.74k	65.83%	301.04us	171.63us	7.30ms	83.08%	298.00us	366.00us	436.00us	731.00us
bin/server_rust_iron	http://127.0.0.1:3000/user/0	0	692034	338555.41	86.46MB	42.30MB	49.73k	34.20k	108.94k	46.43%	340.25us	187.55us	8.02ms	76.21%	332.00us	427.00us	525.00us	0.91ms
bin/server_rust_nickel	http://127.0.0.1:3000/	0	635294	312094.68	78.76MB	38.69MB	106.60k	26.16k	139.32k	51.67%	48.67us	80.92us	8.07ms	99.87%	47.00us	62.00us	72.00us	92.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user	0	608854	300908.77	75.48MB	37.31MB	153.02k	6.29k	173.70k	77.50%	46.96us	23.97us	4.43ms	81.39%	38.00us	50.00us	86.00us	97.00us
bin/server_rust_nickel	http://127.0.0.1:3000/user/0	0	758688	370477.55	119.38MB	58.30MB	95.51k	15.73k	123.40k	56.25%	46.65us	86.10us	8.04ms	99.71%	42.00us	53.00us	64.00us	89.00us
bin/server_rust_rocket	http://127.0.0.1:3000/	0	780129	382597.16	67.70MB	33.20MB	98.11k	21.17k	133.01k	71.25%	65.72us	112.27us	8.07ms	99.70%	56.00us	82.00us	98.00us	127.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user	0	598831	296461.41	51.97MB	25.73MB	150.65k	4.50k	159.10k	65.00%	72.21us	44.34us	12.10ms	85.30%	58.00us	81.00us	139.00us	155.00us
bin/server_rust_rocket	http://127.0.0.1:3000/user/0	0	623223	304948.38	96.28MB	47.11MB	104.48k	15.43k	126.24k	61.67%	74.94us	107.88us	11.67ms	99.67%	64.00us	96.00us	122.00us	155.00us

As expected, C++ was absolutely faster than everything else by a fairly healthy (though not a magnetudal) margin, with Go's fasthttprouter coming up second.

Should I PR it in to use as a good baseline @tbrand and others?

[OvermindDL1]

I added a final listing of 'Rankings' based on the average requests per second, running the above again gives (skipping all the setup/running stuff as it's identical, and as usual a 2 second testing time does not give any real difference from 30 second testing time, so 2s is what I used so I'm not sitting here for a half hour while it runs):

|Path|URL|Errors|Total Requests Count|Total Requests/s|Total Requests Throughput|Total Throughput/s|Req/s Avg|Req/s Std
ev|Req/s Max|Req/s +/-|Latency Avg|Latency Stdev|Latency Max|Latency +/-|50%|75%|90%|99%|
|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|:---|
|bin/server_cpp_evhtp|http://127.0.0.1:3000/|0|1395094|671023.13|85.15MB|40.96MB|87.48k|24.27k|125.13k|57.50%|1.78ms|2.
55ms|29.57ms|88.57%|635.00us|2.03ms|4.77ms|12.59ms|
|bin/server_cpp_evhtp|http://127.0.0.1:3000/user|0|1331146|652502.4|81.25MB|39.83MB|83.43k|24.54k|126.51k|55.62%|1.92ms
|3.00ms|205.95ms|89.27%|679.00us|2.30ms|5.14ms|12.55ms|
|bin/server_cpp_evhtp|http://127.0.0.1:3000/user/0|0|1323210|637653.77|118.62MB|57.16MB|82.88k|25.60k|124.44k|61.25%|1.
76ms|2.34ms|26.02ms|87.82%|658.00us|2.15ms|4.67ms|11.46ms|
|bin/server_crystal_kemal|http://127.0.0.1:3000/|0|102046|49847.64|10.51MB|5.13MB|6.41k|750.52|7.23k|93.75%|19.65ms|5.0
6ms|67.25ms|75.54%|20.16ms|20.59ms|21.77ms|45.71ms|
|bin/server_crystal_kemal|http://127.0.0.1:3000/user|0|82170|40430.31|8.46MB|4.16MB|5.16k|678.14|6.07k|80.62%|23.67ms|4
.16ms|63.25ms|85.56%|23.46ms|23.82ms|25.69ms|40.77ms|
|bin/server_crystal_kemal|http://127.0.0.1:3000/user/0|0|72036|35269.87|9.48MB|4.64MB|4.52k|528.98|5.05k|91.88%|27.55ms
|4.45ms|67.80ms|83.11%|25.85ms|26.26ms|33.70ms|48.19ms|
|bin/server_crystal_raze|http://127.0.0.1:3000/|0|137731|67679.45|8.14MB|4.00MB|8.65k|521.85|9.53k|66.88%|14.00ms|3.17m
s|23.63ms|56.88%|12.25ms|17.04ms|17.63ms|20.74ms|
|bin/server_crystal_raze|http://127.0.0.1:3000/user|0|111396|54934.95|6.59MB|3.25MB|7.00k|745.21|7.78k|91.88%|17.32ms|3
.50ms|221.38ms|67.11%|18.28ms|18.86ms|20.06ms|25.23ms|
|bin/server_crystal_raze|http://127.0.0.1:3000/user/0|0|102461|49862.13|8.99MB|4.37MB|6.44k|383.59|7.07k|80.00%|19.44ms
|3.25ms|41.51ms|74.79%|20.28ms|20.51ms|21.04ms|27.53ms|
|bin/server_crystal_router_cr|http://127.0.0.1:3000/|0|144944|70487.87|8.57MB|4.17MB|9.11k|496.64|10.03k|66.25%|13.73ms
|3.48ms|32.13ms|77.10%|11.87ms|17.14ms|17.75ms|25.18ms|
|bin/server_crystal_router_cr|http://127.0.0.1:3000/user|0|122785|60668.24|7.26MB|3.59MB|7.71k|1.21k|10.00k|71.25%|15.1
7ms|4.82ms|225.68ms|88.06%|14.11ms|18.66ms|20.38ms|26.56ms|
|bin/server_crystal_router_cr|http://127.0.0.1:3000/user/0|0|141745|69016.54|12.44MB|6.06MB|8.90k|1.01k|11.28k|71.25%|1
3.26ms|3.32ms|27.99ms|77.15%|12.02ms|14.07ms|18.83ms|23.58ms|
|bin/server_go_echo|http://127.0.0.1:3000/|0|548500|265442.49|60.68MB|29.36MB|34.22k|6.12k|51.41k|70.00%|3.61ms|10.05ms
|209.73ms|99.21%|2.30ms|4.11ms|6.10ms|12.35ms|
|bin/server_go_echo|http://127.0.0.1:3000/user|0|535638|261031.4|59.26MB|28.88MB|33.51k|4.25k|46.94k|68.75%|4.29ms|12.7
3ms|218.46ms|98.47%|2.27ms|4.36ms|6.60ms|27.21ms|
|bin/server_go_echo|http://127.0.0.1:3000/user/0|0|525061|256538.91|73.11MB|35.72MB|32.76k|5.58k|49.35k|72.50%|4.48ms|1
4.43ms|216.95ms|98.71%|2.34ms|4.37ms|6.42ms|61.52ms|
|bin/server_go_fasthttprouter|http://127.0.0.1:3000/|0|1097978|531179.1|97.38MB|47.11MB|68.61k|14.23k|120.32k|73.12%|2.
05ms|7.95ms|207.97ms|99.10%|1.09ms|1.75ms|3.43ms|9.38ms|
|bin/server_go_fasthttprouter|http://127.0.0.1:3000/user|0|1031467|506227.56|91.48MB|44.90MB|64.50k|14.27k|106.32k|69.3
8%|1.78ms|3.82ms|202.72ms|96.90%|1.17ms|2.00ms|3.49ms|7.71ms|
|bin/server_go_fasthttprouter|http://127.0.0.1:3000/user/0|0|1005476|487158.19|157.26MB|76.19MB|62.86k|18.76k|108.31k|6
8.75%|2.10ms|7.66ms|205.56ms|99.00%|1.10ms|1.93ms|3.61ms|9.74ms|
|bin/server_go_gin|http://127.0.0.1:3000/|0|452591|218616.49|50.07MB|24.18MB|28.25k|3.73k|46.36k|75.62%|4.13ms|7.56ms|2
22.73ms|97.51%|3.12ms|5.14ms|7.43ms|16.04ms|
|bin/server_go_gin|http://127.0.0.1:3000/user|0|451650|216942.3|49.96MB|24.00MB|28.19k|4.03k|41.78k|71.25%|4.35ms|6.57m
s|207.49ms|93.62%|3.03ms|5.25ms|8.51ms|21.22ms|
|bin/server_go_gin|http://127.0.0.1:3000/user/0|0|442026|213831.06|61.55MB|29.77MB|27.60k|4.31k|41.96k|67.50%|5.07ms|13
.23ms|228.19ms|98.17%|3.09ms|5.20ms|7.89ms|30.64ms|
|bin/server_go_gorilla_mux|http://127.0.0.1:3000/|0|515065|250041.75|56.98MB|27.66MB|32.20k|3.90k|45.72k|63.75%|3.51ms|
4.60ms|209.53ms|95.48%|2.73ms|4.53ms|6.47ms|12.57ms|
|bin/server_go_gorilla_mux|http://127.0.0.1:3000/user|0|504362|244060.62|55.80MB|27.00MB|31.51k|4.75k|48.17k|64.38%|4.3
2ms|11.12ms|215.93ms|98.17%|2.62ms|4.64ms|7.12ms|21.48ms|
|bin/server_go_gorilla_mux|http://127.0.0.1:3000/user/0|0|455477|222109.9|63.42MB|30.93MB|28.42k|3.55k|40.67k|70.62%|4.
35ms|10.15ms|231.42ms|98.61%|3.05ms|4.99ms|7.09ms|18.13ms|
|bin/server_go_iris|http://127.0.0.1:3000/|0|619925|301331.9|68.58MB|33.34MB|38.68k|7.55k|67.02k|69.38%|4.35ms|15.35ms|
215.75ms|98.49%|1.99ms|3.71ms|5.95ms|83.36ms|
|bin/server_go_iris|http://127.0.0.1:3000/user|0|595026|287076.76|65.83MB|31.76MB|37.01k|6.13k|63.92k|75.62%|3.34ms|3.3
5ms|212.57ms|89.15%|2.30ms|4.28ms|6.95ms|16.01ms|
|bin/server_go_iris|http://127.0.0.1:3000/user/0|0|578516|277833.09|80.55MB|38.68MB|36.10k|5.79k|54.64k|70.62%|3.50ms|8
.64ms|208.74ms|98.53%|2.25ms|3.88ms|6.16ms|14.54ms|
|bin/server_rust_iron|http://127.0.0.1:3000/|0|786902|380163.34|56.28MB|27.19MB|56.50k|37.41k|117.18k|45.00%|271.93us|2
13.51us|10.29ms|91.86%|254.00us|337.00us|432.00us|0.89ms|
|bin/server_rust_iron|http://127.0.0.1:3000/user|0|712109|348706.38|50.93MB|24.94MB|71.62k|37.40k|112.63k|73.00%|233.93
us|151.66us|8.51ms|84.90%|216.00us|296.00us|355.00us|625.00us|
|bin/server_rust_iron|http://127.0.0.1:3000/user/0|0|689735|335170.19|86.17MB|41.87MB|57.80k|31.32k|109.81k|65.83%|338.
68us|225.26us|11.92ms|85.96%|334.00us|418.00us|508.00us|0.86ms|
|bin/server_rust_nickel|http://127.0.0.1:3000/|0|639506|315462.71|79.28MB|39.11MB|107.28k|33.83k|143.00k|63.33%|47.51us
|49.41us|7.99ms|95.30%|39.00us|48.00us|88.00us|106.00us|
|bin/server_rust_nickel|http://127.0.0.1:3000/user|0|599424|296306.4|74.32MB|36.74MB|150.69k|3.42k|154.48k|85.00%|49.29
us|30.71us|7.65ms|86.98%|41.00us|50.00us|82.00us|92.00us|
|bin/server_rust_nickel|http://127.0.0.1:3000/user/0|0|728491|353094.08|113.94MB|55.22MB|122.23k|7.61k|141.77k|71.67%|4
8.08us|90.67us|9.61ms|99.91%|45.00us|51.00us|57.00us|73.00us|
|bin/server_rust_rocket|http://127.0.0.1:3000/|0|666507|325571.64|57.84MB|28.25MB|111.72k|27.71k|150.06k|58.33%|66.12us
|78.95us|8.04ms|97.82%|62.00us|75.00us|120.00us|153.00us|
|bin/server_rust_rocket|http://127.0.0.1:3000/user|0|605937|298236.53|52.59MB|25.88MB|152.44k|3.89k|161.07k|67.50%|72.8
0us|105.77us|8.08ms|99.86%|66.00us|76.00us|117.00us|136.00us|
|bin/server_rust_rocket|http://127.0.0.1:3000/user/0|0|652278|320427.22|100.77MB|49.50MB|109.44k|21.11k|153.35k|58.33%|
74.28us|151.27us|11.95ms|99.76%|61.00us|93.00us|108.00us|141.00us|

Rankings

Ranking by Average Requests per second:

1. 653726 req/sec : bin/server_cpp_evhtp
2. 508188 req/sec : bin/server_go_fasthttprouter
3. 354679 req/sec : bin/server_rust_iron
4. 321621 req/sec : bin/server_rust_nickel
5. 314745 req/sec : bin/server_rust_rocket
6. 288747 req/sec : bin/server_go_iris
7. 261004 req/sec : bin/server_go_echo
8. 238737 req/sec : bin/server_go_gorilla_mux
9. 216463 req/sec : bin/server_go_gin
10. 66724 req/sec : bin/server_crystal_router_cr
11. 57492 req/sec : bin/server_crystal_raze
12. 41849 req/sec : bin/server_crystal_kemal