A set of Ruby benchmarks for testing Ruby implementations.
C Ruby Emacs Lisp Java CSS JavaScript Other
Latest commit 314ae79 Nov 15, 2013 @rdp rdp Merge pull request #11 from agrimm/fix_spelling
Fix spelling from compliment to complement
Failed to load latest commit information.
benchmarks Fix spelling from compliment to complement Oct 13, 2013
deprecated/deprecated_rbs Fix spelling from compliment to complement Oct 13, 2013
rakelib Merge branch 'master' of git@github.com:acangiano/ruby-benchmark-suite May 18, 2010
.gitignore Fixing accidental push of MegLev specific bench.rake Mar 30, 2009
LICENSE Added Brian Ford's refactoring and new drivers. See the README and Mar 11, 2009
README add a note about stack limit and correct a name of rals benchmarks re… Oct 2, 2011
Rakefile overcome merge difficulty Jun 13, 2009


The Ruby Benchmark Suite is a suite for measuring the performance of Ruby implementations, including micro-benchmarks that focus on core Ruby functionalities, as well as macro-benchmarks that represent a variety of real, common workloads. The project is aimed at providing a useful suite for comparing the performance of the available Ruby implementations and, as a positive side effect, to give VM implementers an additional tool to measure and identity performance related issues.

To obtain the suite, simply run:

    git clone git://github.com/acangiano/ruby-benchmark-suite.git

Benchmarks are run using rake (a copy for convenience is located at benchmarks/rails/dependencies/rake-0.8.7.gem). 

Type "rake -T bench" to see options.


  rake bench
  rake bench:file FILE=benchmarks/micro-benchmarks/bm_app_tak.rb
  rake bench:dir DIR=benchmarks/micro-benchmarks
  rake bench:to_csv

The results are generated within the results/ folder (automatically created for you). Running rake bench:to_csv creates a CSV spreadsheet out of them. This is quite handy for comparing your results over time.

The following example runs demonstrates all possible options -- '/bin/ruby_special' as the VM, 3 iterations instead of the default (i.e., 5), 5 second timeout (per script) instead of the default 300s, and outputs all messages instead of swallowing them (the default).  METER_MEMORY defaults to yes, change it to anything else to disable it (it requires slightly extra processing).

  rake VM=/bin/ruby_special ITERATIONS=3 TIMEOUT=5 VERBOSE=1 METER_MEMORY=yes

if you get "SystemStackError stack level too deep" error as results of some benchmarks, try to increase a stack size:

  ulimit -s 16384

  deprecated     - contains the old, deprecated version of the suite and recently deprecated tests
  benchmarks     - contains the benchmarks themselves
  utils          - contains the infrastructure necessary to run the benchmarks

See utils/README for more details.
Additional output formatters are welcome. For example, a :to_wiki would be very convenient to place the results directly in Confluence or similar Wiki software.

If you want to run a different implementation set the environment variable VM to point to a ruby executable or change this line in rakelib/bench.rake:

  VM = ENV['VM'] || "ruby"

or pass in environment variable on the command line

  rake VM=/path/to/ruby bench


There are a few rails 2.3.x tests located in the benchmarks/rails folder. You may need to install a binary dependency to run them, see benchmarks/rails/README_HOW_TO_RUN_THE_RAILS_TESTS file for instructions for your VM.

There is a mailing list at http://groups.google.com/group/ruby-benchmark-suite