Simple performance testing using ab

When testing performance the main issue is the difference between before/after a change. There is no real success/failure, mainly a reasonable response time, and response time change in percentage.

When testing performance, it is also important not to just execute one request, but to make some kind of load on the server.

Apache Benchmark is a quick and simple testing tool that can be useful for such a quick test.

Notes:
If you need to test an authenticated user you need to follow some steps (TBD) to make use of an existing session.
If you test as an anonymous user, just make sure the caching for anonymous is disabled.

Command line

ab -n <requests> -c <concurrency> <URL>

Requests - number of requests to perform for the benchmarking session.
Concurrency - Number of multiple requests to perform at a time.

Simple example

In this example we test performance of a calendar full of events, showing before/after performance of a performance fix.
Before:

$ ab -n 100 -c 5 http://localhost/openscholar/www/john/calendar
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /openscholar/www/john/calendar
Document Length:        109867 bytes

Concurrency Level:      5
Time taken for tests:   156.580 seconds
Complete requests:      100
Failed requests:        88
   (Connect: 0, Receive: 0, Length: 88, Exceptions: 0)
Write errors:           0
Total transferred:      11047592 bytes
HTML transferred:       10986892 bytes
Requests per second:    0.64 [#/sec] (mean)
Time per request:       7829.024 [ms] (mean)
Time per request:       1565.805 [ms] (mean, across all concurrent requests)
Transfer rate:          68.90 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:  5076 7707 846.9   7726    9293
Waiting:     4995 7497 823.3   7511    9133
Total:       5076 7707 846.9   7726    9293

Percentage of the requests served within a certain time (ms)
  50%   7726
  66%   8126
  75%   8381
  80%   8494
  90%   8681
  95%   9178
  98%   9264
  99%   9293
 100%   9293 (longest request)

After:

carlos@Amir-work:/var/www/openscholar$ ab -n 100 -c 5 http://localhost/openscholar/www/john/calendar
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /openscholar/www/john/calendar
Document Length:        80257 bytes

Concurrency Level:      5
Time taken for tests:   142.306 seconds
Complete requests:      100
Failed requests:        88
   (Connect: 0, Receive: 0, Length: 88, Exceptions: 0)
Write errors:           0
Total transferred:      8086688 bytes
HTML transferred:       8025988 bytes
Requests per second:    0.70 [#/sec] (mean)
Time per request:       7115.308 [ms] (mean)
Time per request:       1423.062 [ms] (mean, across all concurrent requests)
Transfer rate:          55.49 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.1      0       8
Processing:  4968 6960 1001.8   6713   10137
Waiting:     4030 6769 1019.5   6590    9931
Total:       4968 6960 1001.7   6721   10137

Percentage of the requests served within a certain time (ms)
  50%   6721
  66%   7105
  75%   7280
  80%   7694
  90%   8507
  95%   8930
  98%   9753
  99%  10137
 100%  10137 (longest request)

Calculation:

To calculate the difference betweeen the tests you need to get the mean time per request. In our case the "before" result was 1565.805 ms and the "after" result was 1423.062 ms. Calculating the difference in percentage we do

(1565-1423) / 1565 * 100 =~ 9%

This means the performance is better in 9%.