hey slow compared to Apache Bench

[ossie-git]

Hi,

I just ran into hey and really liked it (it handles SSL + reporting a lot better than AB does). However, I noticed that running the exact same test using both tools for non-SSL sites shows that AB is a lot more performant. Any ideas on how to make hey perform as well? Here are the two runs straight in a row. I'm using the latest version of both Go + hey. Thanks

root@jessie:~#` ab -n 10000 -c 1000 -k http://localhost/
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        Apache/2.4.10
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        10701 bytes

Concurrency Level:      1000
Time taken for tests:   0.489 seconds
Complete requests:      10000
Failed requests:        50
   (Connect: 0, Receive: 0, Length: 50, Exceptions: 0)
Keep-Alive requests:    9950
Total transferred:      109560048 bytes
HTML transferred:       106474950 bytes
Requests per second:    20458.43 [#/sec] (mean)
Time per request:       48.880 [ms] (mean)
Time per request:       0.049 [ms] (mean, across all concurrent requests)
Transfer rate:          218889.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   6.0      0      40
Processing:     0   24  25.5     23     252
Waiting:        0   14  25.5     11     239
Total:          0   26  29.7     23     291

Percentage of the requests served within a certain time (ms)
  50%     23
  66%     23
  75%     24
  80%     24
  90%     25
  95%     36
  98%     77
  99%    267
 100%    291 (longest request)

# $GOPATH/bin/hey -n 10000 -c 1000 http://localhost/
Summary:
  Total:        3.3780 secs
  Slowest:      3.2212 secs
  Fastest:      0.0007 secs
  Average:      0.2147 secs
  Requests/sec: 2960.3583

Response time histogram:
  0.001 [1]     |
  0.323 [8671]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.645 [818]   |∎∎∎∎
  0.967 [137]   |∎
  1.289 [16]    |
  1.611 [99]    |
  1.933 [13]    |
  2.255 [24]    |
  2.577 [129]   |∎
  2.899 [6]     |
  3.221 [86]    |

Latency distribution:
  10% in 0.0098 secs
  25% in 0.0460 secs
  50% in 0.0982 secs
  75% in 0.1927 secs
  90% in 0.4126 secs
  95% in 0.6554 secs
  99% in 2.3748 secs

Details (average, fastest, slowest):
  DNS+dialup:    0.0580 secs, 0.0000 secs, 3.1597 secs
  DNS-lookup:    0.0040 secs, 0.0000 secs, 0.2135 secs
  req write:     0.0068 secs, 0.0000 secs, 0.4434 secs
  resp wait:     0.0847 secs, 0.0004 secs, 1.8325 secs
  resp read:     0.0628 secs, 0.0001 secs, 0.9704 secs

Status code distribution:
  [200] 10000 responses

[mdakin]

@ossie-git On my machine, running against a 10K static page, hey could easily handle >20K request per second on plain http.

Could you test with the latest go + hey version? Also maybe give details about the machine you run and the file you serve?

[mdakin]

I can't reproduce this, on my local server with http hey is quite fast. Feel free to reopen the bug if you still see slowness.