First pass at HTTP req/sec benchmark

[ry]

Tests against node.

Ref #977
cc @alexhultman

[ghost]

you can remove this line it is inferred

[ry]

fixed.

[ghost]

Nice to see this taking shape, but you need to keep deno connections open

[ry]

Fortunately/unfortunately I'm seeing that Deno is faster than Node now at this benchmark:

> ./tools/http_benchmark.py out/release/deno
http_benchmark testing DENO.
Listening on 127.0.0.1:4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    32.31ms  128.08ms 861.36ms   94.27%
    Req/Sec     2.24k   438.85     2.65k    92.47%
  41665 requests in 10.01s, 2.03MB read
  Socket errors: connect 0, read 41664, write 0, timeout 0
Requests/sec:   4162.71
Transfer/sec:    207.32KB

http_benchmark testing NODE.
port 4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.47ms  524.32us  14.80ms   92.22%
    Req/Sec     3.05k   511.42     3.61k    86.79%
  32230 requests in 10.00s, 3.84MB read
Requests/sec:   3221.81
Transfer/sec:    393.29KB

So, I will update it to do keep-alive.

[ry]

Using keep-alive (assuming it is doing the connection handling properly) I am also seeing deno faster than node:

> ./tools/http_benchmark.py out/release/deno
http_benchmark testing DENO.
Compiling /Users/rld/src/deno/tests/http_bench.ts
Listening on 127.0.0.1:4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   699.92us    0.96ms  21.33ms   96.02%
    Req/Sec     8.84k     0.98k    9.88k    89.00%
  175851 requests in 10.00s, 12.58MB read
Requests/sec:  17583.11
Transfer/sec:      1.26MB

http_benchmark testing NODE.
port 4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   696.01us  295.58us  12.66ms   96.08%
    Req/Sec     7.33k     1.05k    7.87k    92.57%
  147273 requests in 10.10s, 18.26MB read
Requests/sec:  14581.40
Transfer/sec:      1.81MB

[ry]

On travis, I get something like this (previous benchmarks were from my mac laptop)

Listening on 127.0.0.1:4544
third_party/wrk/linux/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.93ms    1.51ms  20.56ms   94.59%
    Req/Sec     7.85k     1.04k   15.92k    87.06%
  157049 requests in 10.10s, 11.23MB read
Requests/sec:  15550.37
Transfer/sec:      1.11MB
http_benchmark testing NODE.
port 4544
third_party/wrk/linux/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   588.21us  413.42us  11.46ms   96.68%
    Req/Sec     9.02k     1.33k   18.88k    93.53%
  180468 requests in 10.10s, 22.37MB read
Requests/sec:  17868.22
Transfer/sec:      2.22MB

seems good. will land now.

[ghost]

Nice to see this coming together, however some reminders:

Node.js score 2x if you replace the HTTP module with a similarly simple net placeholder implementation:

const net = require('net')

let k = Buffer.from("HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\nHello World!");

net.createServer(function (socket) {
  socket.on('data', function(data) {
    socket.write(k);
  })
}).listen(3000)

For me this goes from 20k to 40k.

Also, remember that Deno is not really strictly single-threaded in CPU-time utilization but more of a 1.6x consumer. Normalizing the numbers with this in mind, together with the much simpler Node.js net implementaiton, Deno is not any more efficient than Node.js in fact they differ very much.

[ry]

@alexhultman yep - Deno is clearly slower. We will pick the optimization tree now.

[ghost]

👍