Which is the fastest?

This project aims to be a load benchmarking suite, no more, no less

Measuring response times (routing times) for each framework (middleware).

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Results are not production-ready yet

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Additional purposes :

• Helping decide between languages, depending on use case
• Learning languages, best practices, devops culture ...
• Having fun ❤️

Requirements

• Crystal as built-in tools are made in this language
• Docker as frameworks are isolated into containers
• wrk as benchmarking tool, >= 4.1.0
• postgresql to store data, >= 10

ℹ️ℹ️ℹ️ℹ️ℹ️

⚠️ On OSX you need docker-machine to use docker containerization

brew install docker-machine
docker-machine create default
eval $(docker-machine env default)

ℹ️ℹ️ℹ️ℹ️ℹ️

Usage

• Install all dependencies

shards install

• Build internal tools

shards build

• Create and initialize the database

createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sql

Docker can be used to set up the database:

docker run -it --rm -d \
  -p 5432:5432 \
  -e POSTGRES_DB=benchmark \
  -e POSTGRES_HOST_AUTH_METHOD=trust \
  -v /tmp/pg-data:/var/lib/postgresql/data \
  --name pg postgres:12-alpine

Wait several seconds for the container to start, then inject the dump:

docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"

After creating the database, export its URL:

export DATABASE_URL="postgresql://postgres@localhost/benchmark"

• Make configuration

bin/make config

• Build containers

jobs are either languages (example : crystal) or frameworks (example : router.cr)

bin/neph [job1] [job2] [job3] ...

• Export all results readme

bin/db to_readme

Results

ℹ️ Updated on 2020-06-21 ℹ️

Benchmarking with wrk

• Threads : 8
• Timeout : 8
• Duration : 15s (seconds)

ℹ️ Sorted by max req/s on concurrency 64 ℹ️

	Language	Framework	Speed (64)	Speed (256)	Speed (512)
1	javascript (13.14)	nanoexpress-pro (1.11)	190 210	205 607	206 446
2	nim (1.2)	httpbeast (0.2)	189 668	205 134	205 541
3	nim (1.2)	whip (0.2)	183 142	195 577	195 507
4	javascript (13.14)	sifrr (0.0)	178 366	192 478	193 568
5	kotlin (1.3)	kooby (2.8)	177 099	198 651	202 836
6	go (1.14)	fasthttp (1.14)	169 416	181 426	187 212
7	php (7.4)	simps (1.0)	167 536	174 608	177 752
8	go (1.14)	router (1.2)	164 609	175 465	181 623
9	go (1.14)	atreugo (11.3)	164 598	175 476	180 878
10	go (1.14)	fasthttprouter (0.1)	163 365	175 045	180 335
11	go (1.14)	gorouter-fasthttp (4.4)	162 895	172 802	178 879
12	go (1.14)	fiber (1.12)	162 442	167 506	168 400
13	go (1.14)	gearbox (1.0)	160 016	172 591	178 141
14	crystal (0.34)	toro (0.4)	158 219	165 480	162 092
15	crystal (0.34)	router.cr (0.2)	157 937	164 624	161 434
16	java (8)	jooby (2.8)	153 520	173 417	177 082
17	crystal (0.34)	spider-gazelle (3.0)	152 719	158 616	155 730
18	java (8)	rapidoid (5.5)	152 167	167 037	165 565
19	nim (1.2)	jester (0.4)	151 010	161 126	162 122
20	php (7.4)	workerman (4.0)	150 081	161 305	161 282
21	crystal (0.34)	kemal (0.26)	145 543	152 512	149 125
22	crystal (0.34)	amber (0.34)	138 946	144 090	140 238
23	rust (1.44)	actix (2.0)	138 586	138 184	138 170
24	crystal (0.34)	orion (2.3)	133 062	136 725	133 301
25	crystal (0.34)	athena (0.8)	127 898	128 886	122 717
26	c (99)	kore (3.3)	123 538	116 655	136 825
27	go (1.14)	rte (0.0)	115 709	115 805	121 317
28	java (8)	act (1.8)	115 251	128 598	127 793
29	go (1.14)	httprouter (1.3)	114 193	114 604	119 472
30	go (1.14)	gin (1.6)	111 598	116 058	120 280
31	go (1.14)	echo (4.1)	110 748	114 235	117 054
32	go (1.14)	chi (4.1)	109 629	108 842	113 195
33	go (1.14)	gorouter (4.4)	109 628	113 863	117 755
34	go (1.14)	aero (1.3)	108 004	109 008	112 906
35	go (1.14)	violetear (7.0)	106 752	106 694	111 666
36	go (1.14)	goroute (0.0)	105 843	104 263	108 225
37	go (1.14)	kami (2.2)	103 905	109 277	112 391
38	go (1.14)	gorilla-mux (1.7)	103 820	101 193	105 252
39	go (1.14)	webgo (4.1)	103 795	101 077	108 098
40	go (1.14)	beego (1.12)	103 307	106 635	110 731
41	csharp (8.0)	aspnetcore (3.1)	97 694	103 022	104 817
42	ruby (2.7)	agoo (2.12)	96 693	124 060	129 738
43	go (1.14)	air (0.17)	90 966	91 325	94 777
44	php (7.4)	one (2.0)	90 944	96 644	97 403
45	javascript (13.14)	polkadot (1.0)	90 846	100 103	100 351
46	php (7.4)	hyperf (1.1)	89 561	95 442	96 182
47	c (11)	agoo-c (0.7)	88 321	181 527	177 078
48	cpp (14/17)	drogon (1.0)	88 265	93 790	95 909
49	fsharp (4.7)	falco (1.1)	84 704	91 083	91 436
50	javascript (13.14)	0http (2.5)	84 193	92 298	94 726
51	fsharp (4.7)	giraffe (4.1)	83 171	87 121	89 800
52	kotlin (1.3)	ktor (1.2)	82 358	97 169	99 574
53	elixir (1.1)	cowboy_stream (2.8)	82 053	82 436	79 555
54	swift (5.2)	perfect (3.1)	81 701	89 649	94 943
55	javascript (13.14)	polka (0.5)	81 605	86 359	85 361
56	go (1.14)	gf (1.13)	81 197	87 334	89 158
57	javascript (13.14)	rayo (1.3)	80 090	84 318	83 526
58	javascript (13.14)	restana (4.6)	78 919	86 121	86 945
59	scala (2.13)	akkahttp (10.1)	78 499	84 614	82 155
60	csharp (8.0)	carter (5.1)	78 082	75 417	66 939
61	fsharp (4.7)	frank (6.1)	77 086	92 926	85 499
62	fsharp (4.7)	saturn (0.14)	72 791	66 108	57 728
63	javascript (13.14)	muneem (2.4)	71 262	76 677	74 929
64	java (8)	javalin (3.8)	70 483	75 759	75 543
65	go (1.14)	mars (1.0)	68 621	68 270	71 745
66	javascript (13.14)	foxify (0.1)	66 571	70 081	69 108
67	haskell (8.8)	scotty (0.12)	64 688	66 701	72 310
68	python (3.8)	falcon (2.0)	64 260	72 978	74 095
69	javascript (13.14)	fastify (2.15)	63 475	66 568	67 715
70	java (8)	spring-boot (2.3)	62 544	68 564	67 600
71	java (8)	micronaut (1.2)	60 973	67 724	67 694
72	javascript (13.14)	koa (2.12)	60 253	61 197	59 861
73	javascript (13.14)	nestjs-fastify (7.2)	60 045	64 017	62 545
74	javascript (13.14)	iotjs-express (0.0)	58 785	60 546	59 004
75	python (3.8)	bottle (0.12)	57 098	57 915	59 133
76	php (7.4)	siler-swoole (1.7)	56 292	74 506	75 369
77	clojure (1.1)	coast (1.0)	55 776	56 660	56 864
78	python (3.8)	apidaora (0.26)	55 441	62 100	62 009
79	fsharp (4.7)	websharper (4.6)	55 372	53 444	49 534
80	swift (5.2)	kitura-nio (2.9)	55 230	52 484	52 657
81	swift (5.2)	kitura (2.9)	54 824	55 250	54 593
82	rust (1.44)	nickel (0.11)	54 464	55 101	54 404
83	javascript (13.14)	express (4.17)	53 914	55 605	54 953
84	javascript (13.14)	feathersjs (4.5)	52 859	55 640	54 614
85	scala (2.13)	http4s (0.21)	52 142	53 851	51 609
86	java (8)	spring-framework (5.2)	51 806	59 926	59 421
87	rust (1.44)	gotham (0.4)	51 034	57 763	59 222
88	swift (5.2)	vapor (4.10)	50 819	52 163	52 124
89	elixir (1.1)	cowboy (2.8)	50 340	50 458	50 350
90	python (3.8)	asgineer (0.7)	50 312	57 377	57 964
91	python (3.8)	pyramid (1.1)	49 400	50 586	50 501
92	javascript (13.14)	moleculer (0.14)	48 698	50 368	48 835
93	python (3.8)	blacksheep (0.2)	46 789	53 771	52 261
94	cpp (11)	evhtp (1.2)	46 286	45 436	45 843
95	python (3.8)	sanic (20.3)	45 953	48 984	48 914
96	python (3.8)	starlette (0.13)	43 895	47 924	47 908
97	javascript (13.14)	hapi (19.1)	43 222	45 447	44 935
98	elixir (1.1)	plug (1.10)	42 260	42 656	42 907
99	scala (2.13)	play (2.8)	42 072	41 851	41 947
100	python (3.8)	hug (2.6)	41 748	42 838	41 074
101	javascript (13.14)	nestjs-express (7.2)	41 659	43 097	42 527
102	javascript (13.14)	restify (8.5)	40 327	42 437	41 818
103	php (7.4)	imi (1.2)	40 311	38 545	40 528
104	php (7.4)	sw-fw-less (preview)	40 304	42 823	42 796
105	ruby (2.7)	syro (3.2)	38 736	40 155	38 297
106	ruby (2.7)	hanami-api (0.1)	37 776	39 256	36 761
107	php (7.4)	yii-swoole (2.0)	37 628	43 005	40 625
108	ruby (2.7)	roda (3.33)	37 353	38 271	36 541
109	python (3.8)	emmett (2.0)	37 324	42 839	41 373
110	elixir (1.1)	phoenix (1.5)	36 476	36 811	37 056
111	clojure (1.1)	luminus (1.0)	35 166	33 549	35 465
112	csharp (8.0)	simplify.web (4.1)	34 503	34 677	33 566
113	php (7.4)	swoft (2.0)	33 681	38 049	38 476
114	ruby (2.7)	cuba (3.9)	32 631	33 519	32 535
115	dart (2.8)	aqueduct (3.3)	31 037	30 867	30 948
116	python (3.8)	fastapi (0.58)	30 625	32 725	32 376
117	haskell (8.8)	servant (0.17)	30 348	29 320	28 386
118	nim (1.2)	rosencrantz (0.4)	29 131	25 828	23 024
119	fsharp (4.7)	suave (2.5)	28 725	29 751	32 409
120	ruby (2.7)	rack-routing (0.0)	28 607	28 286	27 475
121	python (3.8)	responder (2.0)	28 416	30 725	30 182
122	nim (1.2)	akane (0.1)	27 996	24 879	22 111
123	ruby (2.7)	rack_app (7.6)	27 643	28 017	27 241
124	python (3.8)	clastic (19.9)	26 526	26 647	24 965
125	python (3.8)	molten (1.0)	26 079	26 432	26 026
126	go (1.14)	macaron (1.3)	25 917	28 433	28 426
127	ruby (2.7)	camping (2.1)	24 541	24 428	23 801
128	python (3.8)	aiohttp (3.6)	24 075	27 024	27 135
129	rust (1.44)	iron (0.6)	23 875	23 978	23 986
130	javascript (13.14)	turbo_polka (0.3)	23 389	22 448	21 427
131	python (3.8)	flask (1.1)	23 161	23 991	22 613
132	python (3.8)	masonite (2.3)	20 394	21 303	20 002
133	go (1.14)	tango (0.6)	17 738	17 917	17 875
134	java (8)	blade (2.0)	16 955	18 687	19 041
135	php (7.4)	spiral (2.4)	16 677	17 822	18 386
136	ruby (2.7)	sinatra (2.0)	16 078	15 848	15 821
137	go (1.14)	gramework (1.7)	15 087	15 446	15 397
138	ruby (2.7)	grape (1.3)	15 017	14 901	14 797
139	php (7.4)	chubbyphp-roadrunner (2.8)	13 976	15 158	15 526
140	dart (2.8)	start (0.4)	13 511	13 427	12 486
141	javascript (13.14)	sails (1.2)	12 670	12 935	12 975
142	python (3.8)	quart (0.12)	12 662	12 552	11 872
143	crystal (0.34)	runcobo (0.2)	12 232	11 997	11 753
144	ruby (2.7)	flame (4.18)	11 811	11 500	11 506
145	ruby (2.7)	hanami (1.3)	11 005	10 907	10 865
146	php (7.4)	fastsitephp (1.2)	10 983	10 878	10 636
147	swift (5.2)	swifter (1.4)	10 845	10 940	11 089
148	python (3.8)	tornado (6.0)	9 534	9 390	9 304
149	php (7.4)	ubiquity (2.3)	8 742	8 167	7 716
150	python (3.8)	cherrypy (18.6)	8 508	8 711	8 790
151	python (3.8)	django (3.0)	8 307	8 138	8 307
152	python (3.8)	tonberry (0.2)	7 782	7 548	7 132
153	php (7.4)	one-fpm (2.0)	7 739	7 653	7 762
154	php (7.4)	phalcon (4.0)	7 514	7 487	7 508
155	php (7.4)	siler (1.7)	7 466	7 435	7 407
156	php (7.4)	hamlet (3.2)	7 238	7 004	7 029
157	php (7.4)	ice (1.5)	6 422	6 470	6 512
158	crystal (0.34)	onyx (0.5)	5 861	6 038	6 074
159	php (7.4)	chubbyphp (2.8)	5 743	5 765	5 882
160	php (7.4)	slim (4.5)	4 719	4 778	4 878
161	php (7.4)	lumen (7.1)	4 602	4 628	4 707
162	php (7.4)	nette (3.0)	4 414	4 482	4 632
163	php (7.4)	yii (2.0)	4 189	4 274	4 330
164	ruby (2.7)	rails (6.0)	4 041	3 841	3 821
165	julia (1.5)	merly (0.2)	3 472	7 987	6 099
166	perl (5.3)	dancer2 (2.0)	3 078	3 177	1 333
167	php (7.4)	symfony (5.1)	3 049	3 122	3 176
168	php (7.4)	mezzio (3.2)	2 852	2 943	3 020
169	v (0.1)	vape (0.3)	2 770	2 608	1 331
170	python (3.8)	cyclone (1.3)	2 220	2 254	2 235
171	python (3.8)	klein (20.6)	1 553	1 575	1 576
172	python (3.8)	nameko (2.12)	1 546	1 512	1 498
173	crystal (0.34)	lucky (0.21)	1 450	1 468	1 467
174	php (7.4)	laminas (3.1)	1 392	1 421	1 436
175	php (7.4)	basicphp (0.9)	722	436	2 086
176	php (7.4)	laravel (7.16)	719	160	2 219

How to contribute ?

In any way you want ...

• Request a framework addition
• Report a bug (on any implementation)
• Suggest an idea
• ...

Any kind of idea is ❤️

Contributors

• Taichiro Suzuki - Author | Maintainer
• OvermindDL1 - Maintainer
• Marwan Rabbâa - Maintainer