-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
83 additions
and
62 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,76 +1,97 @@ | ||
Ab Crunch uses Apache Bench to run various strategies for load testing web projects, | ||
and provides rake tasks for analyzing performance and enforcing performance | ||
standards on CI. | ||
Ab Crunch | ||
|
||
The idea behind Ab Crunch is that basic performance metrics and standards should | ||
be effortless, first-class citizens in the development process, with frequent visibility | ||
and immediate feedback when performance issues are introduced. | ||
|
||
Other tools exist for measuring performance, but we found that they had some drawbacks: | ||
- Not easily integrated into routine development practices, such as automated testing and CI | ||
- Take a long time to set up. | ||
- Take a long time to use. | ||
|
||
We wanted a tool that, while simple, was valid enough to surface basic performance | ||
issues and fast/easy enough to use throughout all our projects. | ||
|
||
Ab Crunch uses Apache Bench to run various strategies for load testing web sites. | ||
It generates rake tasks for running all or some of our tests. These can be configured | ||
to be just informational, or to fail when specified standards are not met. The rake | ||
tasks can then be added to our Continuous Integration (CI) builds, so builds fail when | ||
performance degrades. | ||
|
||
Credits | ||
Chris Lichti, Author | ||
Aaron Hopkins, adviser / contributed strategies | ||
John Williams, adviser / contributed strategies | ||
|
||
Prerequisites | ||
|
||
Must have Apache Bench installed and 'ab' on your path | ||
Must have Apache Bench installed and 'ab' on your path | ||
|
||
Quick Start Guide | ||
|
||
To see some immediate action, require the gem, and run 'rake abcrunch:example' | ||
|
||
Now to use it on your own pages: | ||
|
||
First, define the pages you want to test, and (optionally), the performance | ||
requirements you want them to meet. If you exclude any requirements, your | ||
load test will be informational only, and won't log or raise any errors | ||
based on performance standards. | ||
|
||
For Example: | ||
|
||
@load_test_page_sets = { | ||
:production => [ | ||
{ | ||
:name => "Google home page", | ||
:url => "http://www.google.com/", | ||
:min_queries_per_second => 20, | ||
:max_avg_response_time => 1000, | ||
}, | ||
{ | ||
:name => "Facebook home page", | ||
:url => "http://www.facebook.com/", | ||
} | ||
], | ||
:staging => [ | ||
{ | ||
:name => "Github home page", | ||
:url => "http://www.github.com/", | ||
:max_avg_response_time => 1000, | ||
} | ||
] | ||
} | ||
|
||
require 'abcrunch' | ||
AbCrunch::Config.page_sets = @load_test_page_sets | ||
|
||
In Rails, you can do this in your development and test environments. | ||
|
||
Once you've configured Ab Crunch, you can run rake tasks to load test your pages, like this: | ||
|
||
rake abcrunch:load_test:staging | ||
- or - | ||
rake abcrunch:load_test:all | ||
To see some immediate action, require the gem, and run 'rake abcrunch:example' | ||
|
||
Now to use it on your own pages: | ||
|
||
First, define the pages you want to test, and (optionally), the performance | ||
requirements you want them to meet. If you exclude any requirements, your | ||
load test will be informational only, and won't log or raise any errors | ||
based on performance standards. | ||
|
||
For Example: | ||
|
||
@load_test_page_sets = { | ||
:production => [ | ||
{ | ||
:name => "Google home page", | ||
:url => "http://www.google.com/", | ||
:min_queries_per_second => 20, | ||
:max_avg_response_time => 1000, | ||
}, | ||
{ | ||
:name => "Facebook home page", | ||
:url => "http://www.facebook.com/", | ||
} | ||
], | ||
:staging => [ | ||
{ | ||
:name => "Github home page", | ||
:url => "http://www.github.com/", | ||
:max_avg_response_time => 1000, | ||
} | ||
] | ||
} | ||
|
||
require 'abcrunch' | ||
AbCrunch::Config.page_sets = @load_test_page_sets | ||
|
||
In Rails, you can do this in your development and test environments. | ||
|
||
Once you've configured Ab Crunch, you can run rake tasks to load test your pages, like this: | ||
|
||
rake abcrunch:load_test:staging | ||
- or - | ||
rake abcrunch:load_test:all | ||
|
||
Configuring Pages | ||
|
||
:name - (required) User-friendly name for the page. | ||
:url - (required) Url to test. Can be a string or a Proc. Proc example: | ||
:url => proc do | ||
"http://www.google.com/?q=#{['food','coma','weirds','code'][rand(4)]}" | ||
end, | ||
:name - (required) User-friendly name for the page. | ||
:url - (required) Url to test. Can be a string or a Proc. Proc example: | ||
:url => proc do | ||
"http://www.google.com/?q=#{['food','coma','weirds','code'][rand(4)]}" | ||
end, | ||
|
||
Performance requirements (will raise so CI builds break when requirements fail) | ||
:min_queries_per_second - page must support at least this many QPS | ||
:max_avg_response_time - latency for the page cannot go higher than this | ||
Performance requirements (will raise so CI builds break when requirements fail) | ||
:min_queries_per_second - page must support at least this many QPS | ||
:max_avg_response_time - latency for the page cannot go higher than this | ||
|
||
Other Options | ||
:num_requests - how many requests to make during each (of many) runs [Default: 50] | ||
:max_latency - global maximum latency (in ms) considered to be acceptable [Default: 1000] | ||
:max_degradation_percent - global max percent latency can degrade before being considered unacceptable [Default: 0.5 (iow 50%)] | ||
Other Options | ||
:num_requests - how many requests to make during each (of many) runs [Default: 50] | ||
:max_latency - global maximum latency (in ms) considered to be acceptable [Default: 1000] | ||
:max_degradation_percent - global max percent latency can degrade before being considered unacceptable [Default: 0.5 (iow 50%)] | ||
|
||
|
||
KNOWN GOTCHA: Apache Bench does not like urls that just end with the domain. For example: | ||
http://www.google.com is BAD, but | ||
http://www.google.com/ is fine, for reasons surpassing understanding. | ||
...so for root level urls, be sure to add a trailing slash. | ||
http://www.google.com is BAD, but | ||
http://www.google.com/ is fine, for reasons surpassing understanding. | ||
...so for root level urls, be sure to add a trailing slash. |