Speedup Test::Unit + RSpec + Cucumber by running parallel on multiple CPUs (or cores).
ParallelTests splits tests into even groups(by number of tests or runtime) and runs each group in a single process with its own database.

upgrading from 0.6 ?

Setup for Rails

RailsCasts episode #413 Fast Tests still using Rails 2?

Install

If you use RSpec: ensure you have >= 2.4

As gem

# add to Gemfile
gem "parallel_tests", :group => :development

OR as plugin

rails plugin install git://github.com/grosser/parallel_tests.git

# add to Gemfile
gem "parallel", :group => :development

Add to `config/database.yml`

ParallelTests uses 1 database per test-process.

Process number	1	2	3
`ENV['TEST_ENV_NUMBER']`	''	'2'	'3'

test:
  database: yourproject_test<%= ENV['TEST_ENV_NUMBER'] %>

Create additional database(s)

rake parallel:create

Copy development schema (repeat after migrations)

rake parallel:prepare

Run!

rake parallel:test          # Test::Unit
rake parallel:spec          # RSpec
rake parallel:features      # Cucumber

rake parallel:test[1] --> force 1 CPU --> 86 seconds
rake parallel:test    --> got 2 CPUs? --> 47 seconds
rake parallel:test    --> got 4 CPUs? --> 26 seconds
...

Test by pattern (e.g. use one integration server per subfolder / see if you broke any 'user'-related tests)

rake parallel:test[^test/unit] # every test file in test/unit folder
rake parallel:test[user]  # run users_controller + user_helper + user tests
rake parallel:test['user|product']  # run user and product related tests

Example output

2 processes for 210 specs, ~ 105 specs per process
... test output ...

843 examples, 0 failures, 1 pending

Took 29.925333 seconds

Run an arbitrary task in parallel

RAILS_ENV=test parallel_test -e "rake my:custom:task"
# or
rake parallel:rake[my:custom:task]

Running things once

# effected by race-condition: first process may boot slower the second
# either sleep a bit or use a lock for example File.lock
ParallelTests.first_process? ? do_something : sleep(1)

at_exit do
  if ParallelTests.first_process?
    ParallelTests.wait_for_other_processes_to_finish
    undo_something
  end
end

Loggers

Even process runtimes

Log test runtime to give each process the same runtime.

Rspec: Add to your .rspec_parallel (or .rspec) :

If installed as plugin: -I vendor/plugins/parallel_tests/lib
--format progress
--format ParallelTests::RSpec::RuntimeLogger --out tmp/parallel_runtime_rspec.log

Test::Unit: Add to your test_helper.rb:

require 'parallel_tests/test/runtime_logger'

RSpec: SummaryLogger

This logger logs the test output without the different processes overwriting each other.

Add the following to your .rspec_parallel (or .rspec) :

If installed as plugin: -I vendor/plugins/parallel_tests/lib
--format progress
--format ParallelTests::RSpec::SummaryLogger --out tmp/spec_summary.log

RSpec: FailuresLogger

This logger produces pasteable command-line snippets for each failed example.

E.g.

rspec /path/to/my_spec.rb:123 # should do something

Add the following to your .rspec_parallel (or .rspec) :

If installed as plugin: -I vendor/plugins/parallel_tests/lib
--format progress
--format ParallelTests::RSpec::FailuresLogger --out tmp/failing_specs.log

Cucumber: FailuresLogger

This logger logs failed cucumber scenarios to the specified file. The filename can be passed to cucumber, prefixed with '@' to rerun failures.

Usage:

cucumber --format ParallelTests::Cucumber::FailuresLogger --out tmp/cucumber_failures.log

Or add the formatter to the parallel: profile of your cucumber.yml:

parallel: --format progress --format ParallelTests::Cucumber::FailuresLogger --out tmp/cucumber_failures.log

To rerun failures:

cucumber @tmp/cucumber_failures.log

Setup for non-rails

gem install parallel_tests
# go to your project dir
parallel_test test/
parallel_rspec spec/
parallel_cucumber features/

use ENV['TEST_ENV_NUMBER'] inside your tests to select separate db/memcache/etc.
Only run selected files & folders:

parallel_test test/bar test/baz/foo_text.rb

Options are:

-n [PROCESSES]                   How many processes to use, default: available CPUs
-p, --pattern [PATTERN]          run tests matching this pattern
    --group-by [TYPE]            group tests by:
      found - order of finding files
      steps - number of cucumber steps
      default - runtime or filesize
-m, --multiply-processes [FLOAT] use given number as a multiplier of processes to run
-s, --single [PATTERN]           Run all matching files in the same process
-i, --isolate                    Do not run any other tests in the group used by --single(-s)
-e, --exec [COMMAND]             execute this code parallel and with ENV['TEST_ENV_NUM']
-o, --test-options '[OPTIONS]'   execute test commands with those options
-t, --type [TYPE]                test(default) / rspec / cucumber
    --serialize-stdout           Serialize stdout output, nothing will be written until everything is done
    --non-parallel               execute same commands but do not in parallel, needs --exec
    --no-symlinks                Do not traverse symbolic links to find test files
    --ignore-tags [PATTERN]      When counting steps ignore scenarios with tags that match this pattern
    --nice                       execute test commands with low priority.
-v, --version                    Show Version
-h, --help                       Show this.

You can run any kind of code in parallel with -e / --execute

parallel_test -n 5 -e 'ruby -e "puts %[hello from process #{ENV[:TEST_ENV_NUMBER.to_s].inspect}]"'
hello from process "2"
hello from process ""
hello from process "3"
hello from process "5"
hello from process "4"

	1 Process	2 Processes	4 Processes
RSpec spec-suite	18s	14s	10s
Rails-ActionPack	88s	53s	44s

TIPS

[RSpec] add a .rspec_parallel to use different options, e.g. no --drb
[RSpec] delete script/spec
[Spork] does not work with parallel_tests
[RSpec] remove --loadby from you spec/*.opts
[RSpec] Instantly see failures (instead of just a red F) with rspec-instafail
[Bundler] if you have a Gemfile then bundle exec will be used to run tests
[Cucumber] add a parallel: foo profile to your config/cucumber.yml and it will be used to run parallel tests
Capybara setup
Sphinx setup
Capistrano setup let your tests run on a big box instead of your laptop
[SQL schema format] use :ruby schema format to get faster parallel:prepare`
export PARALLEL_TEST_PROCESSORS=X in your environment and parallel_tests will use this number of processors by default
[ZSH] use quotes to use rake arguments rake "parallel:prepare[3]"
email_spec and/or action_mailer_cache_delivery
[Memcached] use different namespaces e.g. config.cache_store = ..., :namespace => "test_#{ENV['TEST_ENV_NUMBER']}"

TODO

make tests consistently pass with --order random in .rspec
fix tests vs cucumber >= 1.2 unknown option --format
add integration tests for the rake tasks, maybe generate a rails project ...
add unit tests for cucumber runtime formatter
make windows compatible

Authors

inspired by pivotal labs

Contributors

Michael Grosser
michael@grosser.it
License: MIT

nicsnoek/parallel_tests

ErrorLooks like something went wrong!

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Error
Looks like something went wrong!

Packages