Synthesis

Philosophy

Currently we believe that developers are writing unnecessary dependency wired tests to cover uncertainty about the validity of simulated interactions in their dependency neutral tests. In other words, we cannot be certain that all our simulated interaction based tests ‘join up’. If it were possible to correlate the simulated interactions in our tests, then we should be able to do away with the need to write large numbers of complex, slow and brittle wired tests (apart from those which interact with the boundaries of the SUT).

Synthesis combines lightweight tests to build confidence that the system under test is complete and reduces the need for large, overarching tests.

Installation

 sudo gem i synthesis

Download

Synthesis RubyForge page ( rubyforge.org/projects/synthesis )

Dependencies

Synthesis’s core doesn’t have any dependencies.

When used with the Mocha adapter, it will depend on the Mocha library.

When used with the RSpec adapter, it will depend on the RSpec library.

When used with the Expectations adapter, it will depend on the Expectations library.

Usage

Synthesis can be used through its Rake task. It currently has three supported adapters: Mocha (with Test::Unit), RSpec and Expectations. If adapter is not explicitly specified, the Mocha adapter will be used by default.

By default, Synthesis outputs to STDOUT, but output can be redirected to alternative IO streams.

Synthesis can be setup to ignore certain classes or modules when collecting expectations for verification.

If pattern is not specified, it will default to test/**/*_test.rb

As of version 0.2.0, Synthesis has a DOT formatter which, when used, will output text in the DOT graph description language, producing system visualizations as specified by the simulated interactions in the system’s tests. The output of the DOT formatter can be used with tools like Graphviz( www.graphviz.org/ ).

Usage examples

To use with Test::Unit and Mocha, ignoring Array and Hash:

  require "synthesis/task"

  Synthesis::Task.new do |t|
    t.pattern = 'test/unit/**/*_test.rb'
    t.ignored = [Array, Hash]
  end

To use with RSpec, running all specs in the spec directory:

  require "synthesis/task"

  Synthesis::Task.new do |t|
    t.adapter = :rspec
    t.pattern = 'spec/**/*_spec.rb'
  end

To use with Expectations, redirecting output to a file:

  require "synthesis/task"

  Synthesis::Task.new do |t|
    t.adapter = :expectations
    t.out = File.new "synthesis.test.txt", "a"
  end

To output a DOT graph, first make sure you have sexp_processor installed:

  sudo gem install sexp_processor

Then, to output a file called "synthesis.dot", do (if formatter_out is not specified, the default ouput is STDOUT):

  require "synthesis/task"

  Synthesis::Task.new do |t|
    t.formatter = :dot
    t.formatter_out = "synthesis.dot"
  end

To use Synthesis with Rails:

  require "synthesis/task"

  Synthesis::Task.new do |t|
    RAILS_ENV = "test"
    Rake::Task['environment'].invoke # This loads the Rails environment, which may make your build slower. Use only if needed
    t.pattern = 'test/**/*_test.rb'
  end

Utilities

mock_instance

  require "synthesis/util/mock_instance"
  foo_mock = Foo.mock_instance(arg_one, arg_2)

This is equivalent, but without calling the real initialize, to:

  foo_mock = Foo.new
  Foo.expects(:new).with(arg_one, arg_two).returns(foo_mock)

Or, in the case of RSpec, it is equivalent to:

  foo_mock = Foo.new
  Foo.should_receive(:new).with(arg_one, arg_two).and_return(foo_mock)

Either "mocha_standalone" or "spec/mocks" need to be required before using mock_instance.

Git

Public clone URL: git://github.com/gmalamid/synthesis.git

Known Issues

Reporting the location (filename and line number) of tested/untested expectations doesn’t work as intended with the Expectations adapter.

Contributors

Danilo Sato, Paul Nasrat, Jerome Riga

Discuss

groups.google.com/group/synthesized-testing

Related reading

• Synthesized Testing: A Primer ( nutrun.com/weblog/synthesized-testing-a-primer )
• Confidence as a test code metric ( nutrun.com/weblog/confidence-as-a-test-code-metric )
• Using Synthesis With Test::Unit and Mocha ( nutrun.com/weblog/using-synthesis-with-testunit-and-mocha )
• Using Synthesis With Expectations ( nutrun.com/weblog/using-synthesis-with-expectations )
• Synthesis validates simulated method calls taking method signature arguments into account ( nutrun.com/weblog/synthesis-002 )