HTTPS clone URL
Subversion checkout URL
A set of Ruby benchmarks for testing Ruby implementations.
Fix spelling from compliment to complement
The Ruby Benchmark Suite is a suite for measuring the performance of Ruby implementations, including micro-benchmarks that focus on core Ruby functionalities, as well as macro-benchmarks that represent a variety of real, common workloads. The project is aimed at providing a useful suite for comparing the performance of the available Ruby implementations and, as a positive side effect, to give VM implementers an additional tool to measure and identity performance related issues. GETTING THE SUITE ================= To obtain the suite, simply run: git clone git://github.com/acangiano/ruby-benchmark-suite.git RUNNING THE SUITE ================= Benchmarks are run using rake (a copy for convenience is located at benchmarks/rails/dependencies/rake-0.8.7.gem). Type "rake -T bench" to see options. Examples: rake bench rake bench:file FILE=benchmarks/micro-benchmarks/bm_app_tak.rb rake bench:dir DIR=benchmarks/micro-benchmarks rake bench:to_csv The results are generated within the results/ folder (automatically created for you). Running rake bench:to_csv creates a CSV spreadsheet out of them. This is quite handy for comparing your results over time. The following example runs demonstrates all possible options -- '/bin/ruby_special' as the VM, 3 iterations instead of the default (i.e., 5), 5 second timeout (per script) instead of the default 300s, and outputs all messages instead of swallowing them (the default). METER_MEMORY defaults to yes, change it to anything else to disable it (it requires slightly extra processing). rake VM=/bin/ruby_special ITERATIONS=3 TIMEOUT=5 VERBOSE=1 METER_MEMORY=yes if you get "SystemStackError stack level too deep" error as results of some benchmarks, try to increase a stack size: ulimit -s 16384 DIRECTORY STRUCTURE =================== deprecated - contains the old, deprecated version of the suite and recently deprecated tests benchmarks - contains the benchmarks themselves utils - contains the infrastructure necessary to run the benchmarks See utils/README for more details. Additional output formatters are welcome. For example, a :to_wiki would be very convenient to place the results directly in Confluence or similar Wiki software. NOTES ===== If you want to run a different implementation set the environment variable VM to point to a ruby executable or change this line in rakelib/bench.rake: VM = ENV['VM'] || "ruby" or pass in environment variable on the command line rake VM=/path/to/ruby bench RAILS TESTS =========== There are a few rails 2.3.x tests located in the benchmarks/rails folder. You may need to install a binary dependency to run them, see benchmarks/rails/README_HOW_TO_RUN_THE_RAILS_TESTS file for instructions for your VM. HELP ===== There is a mailing list at http://groups.google.com/group/ruby-benchmark-suite