Deno http server serves around 25-40% less req/sec than that of Node.js #6722

trivikr · 2020-07-12T23:37:40Z

While doing performance comparison between Node.js and Deno, I noticed that Deno ~~performance gets affected when number of connections are increased~~ http server serves around 25-40% less req/sec than that of Node.js.

GitHub repo: https://github.com/trivikr/node-deno-http-benchmarks
UPDATE: The updated load testing benchmarks are posted in #6722 (comment)

System Information

~~Click to expand~~

$ sw_vers                           
ProductName:	Mac OS X
ProductVersion:	10.14.6
BuildVersion:	18G3020

$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz

$ deno --version
deno 1.1.3
v8 8.5.216
typescript 3.9.2

$ node --version
v14.5.0

$ autocannon --version
autocannon v5.0.1
node v14.5.0

with 100 connections

~~Node.js v14.5.0~~

~~Deno v1.1.3~~

with 1000 connections

~~Node.js v14.5.0~~

~~Deno v1.1.3~~

kitsonk · 2020-07-12T23:50:06Z

So does Node.js. It just appears that performance drops off a bit quicker with Deno. We had hoped that a JS/TS based HTTP server was going to give us the performance we wanted, but that hasn't been achievable and it is on the roadmap to port it to Rust. (It is also the easiest way to get HTTP/2 and QUIC as well).

There are some HTTP benchmarks (https://deno.land/benchmarks) that are part of CI pipeline. A load test like this might be a good addition.

trivikr · 2020-07-12T23:56:29Z

is on the roadmap to port HTTP server to Rust

Is there a link to this specific request on roadmap?
On quick search, I could find #5625 which is referred from #2473

trivikr · 2020-07-15T13:35:56Z

Updated benchmarks on deno@1.2.0 with std@0.61.0
Also, the headers were updated to return the headers in deno in trivikr/node-deno-http-benchmarks@43e1810

System Information

Click to expand

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:        20.04
Codename:       focal

$ lscpu | grep "Model name:" | sed -r 's/Model name:\s{1,}//g'
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz

$ deno --version
deno 1.2.0
v8 8.5.216
typescript 3.9.2

$ node -v
v14.5.0

$ autocannon -v
autocannon v5.0.1
node v14.5.0

with 10 connections

Node.js 14.5.0

$ autocannon -c10 localhost:3000
Running 10s test @ http://localhost:3000
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 0 ms │ 0.01 ms │ 0.08 ms │ 10.99 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 32367   │ 32367   │ 44223   │ 45503   │ 43404   │ 3756.58 │ 32366   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.59 MB │ 3.59 MB │ 4.91 MB │ 5.05 MB │ 4.82 MB │ 417 kB  │ 3.59 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

434k requests in 10.06s, 48.2 MB read

Deno 1.2.0

$ autocannon -c10 localhost:3000
Running 10s test @ http://localhost:3000
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 1 ms │ 0.02 ms │ 0.15 ms │ 10.95 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬────────┬──────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%  │ Avg      │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼────────┼──────────┼─────────┼─────────┤
│ Req/Sec   │ 19391   │ 19391   │ 27199   │ 30639  │ 26365.82 │ 3429.79 │ 19389   │
├───────────┼─────────┼─────────┼─────────┼────────┼──────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.15 MB │ 2.15 MB │ 3.02 MB │ 3.4 MB │ 2.93 MB  │ 381 kB  │ 2.15 MB │
└───────────┴─────────┴─────────┴─────────┴────────┴──────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

290k requests in 11.06s, 32.2 MB read

with 100 connections

Node.js v14.5.0

autocannon -c100 localhost:3000
Running 10s test @ http://localhost:3000
100 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 1 ms │ 2 ms │ 3 ms  │ 4 ms │ 1.99 ms │ 0.75 ms │ 30.14 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 34463   │ 34463   │ 46719   │ 47295   │ 45481.6 │ 3724.53 │ 34460   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.83 MB │ 3.83 MB │ 5.19 MB │ 5.25 MB │ 5.05 MB │ 413 kB  │ 3.83 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

455k requests in 10.08s, 50.5 MB read

Deno v1.2.0

$ autocannon -c100 localhost:3000
Running 10s test @ http://localhost:3000
100 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬───────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max       │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼───────────┤
│ Latency │ 2 ms │ 2 ms │ 4 ms  │ 4 ms │ 2.28 ms │ 1.14 ms │ 104.55 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 29887   │ 29887   │ 35839   │ 36319   │ 35258.4 │ 1814.32 │ 29881   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.32 MB │ 3.32 MB │ 3.98 MB │ 4.03 MB │ 3.91 MB │ 201 kB  │ 3.32 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

353k requests in 10.08s, 39.1 MB read

with 1000 connections

Node.js v14.5.0

$ autocannon -c1000 localhost:3000
Running 10s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬──────────┬───────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev    │ Max       │
├─────────┼───────┼───────┼───────┼───────┼──────────┼──────────┼───────────┤
│ Latency │ 22 ms │ 24 ms │ 31 ms │ 59 ms │ 25.54 ms │ 14.58 ms │ 311.06 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴──────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 22079   │ 22079   │ 40831   │ 41151   │ 38663.2 │ 5595.15 │ 22065   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.45 MB │ 2.45 MB │ 4.53 MB │ 4.57 MB │ 4.29 MB │ 621 kB  │ 2.45 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

387k requests in 10.22s, 42.9 MB read

Deno v1.2.0

$ autocannon -c1000 localhost:3000
Running 10s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬───────────┬─────────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev     │ Max         │
├─────────┼───────┼───────┼───────┼───────┼──────────┼───────────┼─────────────┤
│ Latency │ 14 ms │ 31 ms │ 42 ms │ 43 ms │ 39.78 ms │ 239.53 ms │ 10013.98 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴───────────┴─────────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec   │ 19151   │ 19151   │ 23839   │ 24703   │ 23235.2 │ 1612.2 │ 19139   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 2.13 MB │ 2.13 MB │ 2.65 MB │ 2.74 MB │ 2.58 MB │ 179 kB │ 2.12 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘

Req/Bytes counts sampled once per second.

232k requests in 10.23s, 25.8 MB read
69 errors (69 timeouts)

trivikr · 2020-07-16T00:30:21Z

Closing as:

This performance difference was known and called out in v1 announcement blog post under HTTP Server Performance https://deno.land/v1#http-server-performance
Deno is tracking this in benchmarks under "HTTP Server throughput" for realistic 10 keep-alive connections https://deno.land/benchmarks
- load tests can be added for 100 or 1000 connections, but v1 blog post recommends using some different programming language for higher throughput
As per comment by @kitsonk porting http server to rust is in the roadmap Deno http server serves around 25-40% less req/sec than that of Node.js #6722 (comment)

n10000k · 2020-07-25T20:01:09Z

@trivikr @kitsonk We got any timelines/roadmap for http to be ported to Rust, can we also think about http2 when the port is due to happen, this seems to have been forgotten about also.

ry · 2020-07-25T20:11:24Z

@narwy We have not started this work, but we likely will undertake this in the coming weeks/months. I have a specific plan of how to do it. ETA is 3 months.

n10000k · 2020-07-25T20:23:48Z

@ry thanks for the update can't wait 👍 I think also the http3 draft has been released this month or something too, could be worth a read also.

trivikr · 2020-07-26T00:05:12Z

I think also the http3 draft has been released this month or something too, could be worth a read also.

QUIC is in draft-29 and it's not standardized yet.
Details https://datatracker.ietf.org/doc/draft-ietf-quic-http/

Support for HTTP/2, HTTP/3 and QUIC is being tracked in #3995

ynwd · 2020-10-12T06:48:08Z

@narwy We have not started this work, but we likely will undertake this in the coming weeks/months. I have a specific plan of how to do it. ETA is 3 months.

@ry Is there an update on this ?

trivikr changed the title ~~Performance of Deno gets affected with increasing number of connections~~ Deno http server serves around 25-40% less req/sec than that of Node.js Jul 15, 2020

This was referenced Jul 15, 2020

Benchmarks are incorrect fastrodev/fastro#8

Closed

Introduce node_raw and fastify benchmarks for realistic comparisons with drash drashland/drash#322

Closed

trivikr closed this as completed Jul 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deno http server serves around 25-40% less req/sec than that of Node.js #6722

Deno http server serves around 25-40% less req/sec than that of Node.js #6722

trivikr commented Jul 12, 2020 •

edited

Loading

kitsonk commented Jul 12, 2020

trivikr commented Jul 12, 2020

trivikr commented Jul 15, 2020

trivikr commented Jul 16, 2020

n10000k commented Jul 25, 2020

ry commented Jul 25, 2020 •

edited

Loading

n10000k commented Jul 25, 2020

trivikr commented Jul 26, 2020

ynwd commented Oct 12, 2020

Deno http server serves around 25-40% less req/sec than that of Node.js #6722

Deno http server serves around 25-40% less req/sec than that of Node.js #6722

Comments

trivikr commented Jul 12, 2020 • edited Loading

System Information

with 100 connections

with 1000 connections

kitsonk commented Jul 12, 2020

trivikr commented Jul 12, 2020

trivikr commented Jul 15, 2020

System Information

with 10 connections

with 100 connections

with 1000 connections

trivikr commented Jul 16, 2020

n10000k commented Jul 25, 2020

ry commented Jul 25, 2020 • edited Loading

n10000k commented Jul 25, 2020

trivikr commented Jul 26, 2020

ynwd commented Oct 12, 2020

trivikr commented Jul 12, 2020 •

edited

Loading

ry commented Jul 25, 2020 •

edited

Loading