Permalink
Browse files

Only run "adapter" tests for different versions of MiniTest & Test::Unit

* The so-called "adapter" tests are really integration tests in the
sense that they check that Mocha is properly integrated with the
relevant test library.

* Running *all* the tests for all the versions of MiniTest & Test::Unit
was overkill. It's only really the integration between Mocha and the
test library that is likely to differ in these cases, so the new
"adapter" tests are sufficient.

* I think I've made the same changes to both the build-matrix script and
and to .travis.yml - it's clearly a pain to have to maintain both of
these, but fixing that is a job for another day.

* We're still not running tests for the 3 earlier versions of MiniTest,
because they don't work for some reason. I had thought it was something
to do with running all the tests, but now it looks like it might just be
that the TestRunner doesn't work with these versions of MiniTest in
Ruby 1.9.3.

* In running these tests, a minor bug in most of the monkey-patch versions
of Test::Unit integration came to light and these are fixed in this
commit. This is gives me some confidence that there is a decent amount
of coverage in the "adapter" tests.
  • Loading branch information...
1 parent 953650d commit f1ff6475ca2871f2977ab84cabbbfe2adadbbee6 @floehopper floehopper committed Aug 25, 2012
View
@@ -4,36 +4,76 @@ rvm:
- 1.9.3
gemfile:
- Gemfile
- - gemfiles/Gemfile.test-unit.latest
- - gemfiles/Gemfile.test-unit.2.0.3
- - gemfiles/Gemfile.test-unit.2.0.1
- - gemfiles/Gemfile.test-unit.2.0.0
- - gemfiles/Gemfile.minitest.latest
- - gemfiles/Gemfile.minitest.2.11.2
- - gemfiles/Gemfile.minitest.2.11.0
- - gemfiles/Gemfile.minitest.2.3.0
- - gemfiles/Gemfile.minitest.2.0.1
- - gemfiles/Gemfile.minitest.2.0.0
- - gemfiles/Gemfile.minitest.1.4.2
- - gemfiles/Gemfile.minitest.1.4.0
- - gemfiles/Gemfile.minitest.1.3.0
env:
- MOCHA_OPTIONS=debug
matrix:
- exclude:
+ include:
- rvm: 1.9.3
- gemfile: gemfiles/Gemfile.minitest.1.3.0
- env: MOCHA_OPTIONS=debug
+ gemfile: gemfiles/Gemfile.minitest.latest
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
- rvm: 1.9.3
- gemfile: gemfiles/Gemfile.minitest.1.4.0
- env: MOCHA_OPTIONS=debug
+ gemfile: gemfiles/Gemfile.minitest.2.11.2
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
- rvm: 1.9.3
- gemfile: gemfiles/Gemfile.minitest.1.4.2
- env: MOCHA_OPTIONS=debug
- include:
+ gemfile: gemfiles/Gemfile.minitest.2.11.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
- rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.minitest.2.3.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.minitest.2.0.1
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.minitest.2.0.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
gemfile: gemfiles/Gemfile.minitest.latest
env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.2.11.2
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.2.11.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.2.3.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.2.0.1
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.2.0.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.1.4.2
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.1.4.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.minitest.1.3.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=minitest
+ - rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.test-unit.latest
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.test-unit.2.0.3
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.test-unit.2.0.1
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
- rvm: 1.9.3
+ gemfile: gemfiles/Gemfile.test-unit.2.0.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.8.7
gemfile: gemfiles/Gemfile.test-unit.latest
env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.test-unit.2.0.3
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.test-unit.2.0.1
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
+ - rvm: 1.8.7
+ gemfile: gemfiles/Gemfile.test-unit.2.0.0
+ env: MOCHA_OPTIONS=debug MOCHA_RUN_ADAPTER_TESTS=test-unit
View
@@ -48,26 +48,22 @@ def run(gemfile, task = "test")
RUBY_VERSIONS.each do |ruby_version|
execute("rbenv local #{ruby_version}")
+ reset_bundle
+ run("Gemfile")
+ execute("rbenv local --unset")
+end
+
+RUBY_VERSIONS.each do |ruby_version|
+ execute("rbenv local #{ruby_version}")
["test-unit", "minitest"].each do |test_library|
reset_bundle
Dir["gemfiles/Gemfile.#{test_library}.*"].each do |gemfile|
ruby_version_without_patch = ruby_version.split("-")[0]
next if (ruby_version_without_patch == "1.9.3") && EXCLUDED_RUBY_193_GEMFILES.include?(gemfile)
p [ruby_version_without_patch, test_library, gemfile]
+ ENV['MOCHA_RUN_ADAPTER_TESTS'] = test_library
run(gemfile)
end
end
execute("rbenv local --unset")
end
-
-ruby_version = "1.9.3-p125-perf"
-execute("rbenv local #{ruby_version}")
-["test-unit", "minitest"].each do |test_library|
- reset_bundle
- gemfile = "gemfiles/Gemfile.#{test_library}.latest"
- ruby_version_without_patch = ruby_version.split("-")[0]
- p [ruby_version_without_patch, test_library, gemfile]
- ENV['MOCHA_RUN_ADAPTER_TESTS'] = test_library
- run(gemfile)
-end
-execute("rbenv local --unset")
@@ -39,6 +39,8 @@ def run(result)
ensure
begin
run_teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Exception
raise unless handle_exception($!)
end
@@ -39,6 +39,8 @@ def run(result)
ensure
begin
run_teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Exception
raise unless handle_exception($!)
end
@@ -39,6 +39,8 @@ def run(result)
ensure
begin
run_teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Exception
raise unless handle_exception($!)
end
@@ -43,6 +43,8 @@ def run(result)
ensure
begin
run_teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Exception
raise unless handle_exception($!)
end
@@ -39,6 +39,8 @@ def run(result)
ensure
begin
teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Test::Unit::AssertionFailedError => e
add_failure(e.message, e.backtrace)
rescue StandardError, ScriptError
@@ -40,6 +40,8 @@ def run(result)
ensure
begin
teardown
+ rescue Mocha::ExpectationError => e
+ add_failure(e.message, e.backtrace)
rescue Test::Unit::AssertionFailedError => e
add_failure(e.message, e.backtrace)
rescue Exception
@@ -1,6 +1,7 @@
require File.expand_path('../../test_helper', __FILE__)
-require 'mocha/mini_test'
+require "minitest/unit"
+require "mocha"
require "adapters/shared_adapter_tests"
class MiniTestTest < Test::Unit::TestCase
@@ -1,6 +1,7 @@
require File.expand_path('../../test_helper', __FILE__)
-require 'mocha/test_unit'
+require "test/unit"
+require "mocha"
require "adapters/shared_adapter_tests"
class TestUnitTest < Test::Unit::TestCase
@@ -1,5 +1,4 @@
require 'stringio'
-require 'test/unit/testcase'
require 'minitest/unit'
class MiniTestResult
View
@@ -12,24 +12,25 @@ def run_as_test(&block)
end
def run_as_tests(methods = {})
- test_class = Class.new(Test::Unit::TestCase) do
+ base_class = defined?(MiniTest) ? MiniTest::Unit::TestCase : Test::Unit::TestCase
+ test_class = Class.new(base_class) do
methods.each do |(method_name, proc)|
define_method(method_name, proc)
end
end
tests = methods.keys.select { |m| m.to_s[/^test/] }.map { |m| test_class.new(m) }
- if defined?(Test::Unit::TestResult)
- test_result = TestUnitResult.build_test_result
- tests.each do |test|
- test.run(test_result) {}
- end
- else
+ if defined?(MiniTest)
runner = MiniTest::Unit.new
tests.each do |test|
test.run(runner)
end
test_result = MiniTestResult.new(runner, tests)
+ else
+ test_result = TestUnitResult.build_test_result
+ tests.each do |test|
+ test.run(test_result) {}
+ end
end
test_result

0 comments on commit f1ff647

Please sign in to comment.