Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Merging doesn't work with test-unit #45

Open
Ecco opened this Issue · 13 comments

9 participants

Romain Goyet Christoph Olszowka Eric Wanchic J. Pablo Fernández georgeu2000 Fernando Guillen Nikos Dimitrakopoulos Michael Irwin Tony Primerano
Romain Goyet

Rails 3 application. Simple tests (no rspec, no cucumber).

If I add this to my gemfile :

require 'test-unit', '2.3.0'

then merging doesn't work as unit, functional, and integration test results overwrite each-other.

Thing is, I need this for the performance tests to work.

Romain Goyet

For some reason, it seems like ARGV is not passed to SimpleCov::CommandGuesser when using "test-unit 2.3.0"

Romain Goyet

Ok, I figured it out : for some weird reason, the "unit-test" gem alters ARGV !
A very simple fix is to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start, so that the gem hasn't had time to touch ARGV.

Christoph Olszowka
Owner

Oh, ok. Haven't tried test-unit 2+ yet, but if it's the way you're saying that it messes with ARGV, that seems like a pretty bad practice to me :/

It'd be great if you could swing up a little part about test/unit 2 including your workaround as a code example for the README and phase out a pull request. Thanks!

Christoph Olszowka colszowka closed this
Christoph Olszowka colszowka reopened this
Christoph Olszowka
Owner

Actually reopening this as a reminder to add something about this into the README

Eric Wanchic

I am also having this problem

J. Pablo Fernández

I got this issue and I spent a couple of hours trying to figure out what was going on... are my tests not running? no, they are running, is there some old coverage data somewhere else? Until I concluded the merging wasn't working, and then upon inspecting the .yml file, realizing results were not separated by test suit. Yeah, documenting this issue seems important, and fixing it if possible too so that it doesn't happen.

georgeu2000

I am having this issue as well. If I view the HTML file after the unit tests part, the results are different then at the end, after the functionals.

Using: $ rake test

Gemfile:
gem 'test-unit', '1.2.3'

I see the fix is: "to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start"
OK, I added SimpleCov.command_name to test_helper.rb. The first 3 lines are now:
require 'simplecov'
SimpleCov.start 'rails'
SimpleCov.command_name

But I am still seeing the same symptom. How can I put the results of both units and functionals in one report? Is it possible to create a rake task that would do it?

Also, I am running test-unit 1.2.3, so I don't know if this is an issue with just test-unit 2?

Otherwise, seems to work great. Thank you!

georgeu2000

Still seeing this issue with 0.5.0. When using rake test, the functional tests results overwrite the unit test results.

Updated to rake 0.9.2.

OK, after working on this for 12 hours, I have narrowed it down to 2 or 3 gems. However, these gems do not always cause the problem and the problem does not disappear unless all 3 are removed:
gem 'mocha'
gem 'redgreen'
gem 'test-unit', '1.2.3'

How can you save the results from a test suite with a given name, and manually merge the results of 2 test suites?

Fernando Guillen

Same problem with test-unit 2.4.0

As Ecco said, it is the test-unit who is eating the ARGV content.

A fix for test-unit 2.4.0 could be:

~/Downloads/test-unit(master) $ git diff --no-prefix 
diff --git lib/test/unit/autorunner.rb lib/test/unit/autorunner.rb
index 7a31aa1..029a4b5 100644
--- lib/test/unit/autorunner.rb
+++ lib/test/unit/autorunner.rb
@@ -146,7 +146,7 @@ module Test
       def process_args(args = ARGV)
         begin
           args.unshift(*@default_arguments)
-          options.order!(args) {|arg| @to_run << arg}
+          options.order!(args.dup) {|arg| @to_run << arg}
         rescue OptionParser::ParseError => e
           puts e
           puts options

I let it here just in case it is helpful for someone, I'll try to comunicate with the test-unit maintainers but maybe I'll forget.

Nikos Dimitrakopoulos

Thanks @fguillen for the tip. Sent a pull request to the test-unit people (test-unit/test-unit#12).

Christoph Olszowka
Owner

Obviously the test-unit guys merged this into Test/Unit, I suspect by release date that v2.4.3+ should have this resolved. Please re-open if there is still trouble.

Thanks a lot for taking care of this guys!

Christoph Olszowka colszowka closed this
Michael Irwin

This problem still exists w/ test-unit 2.4.5 and simplecov 0.5.4.

Tony Primerano

yeah, still broken..

The workaround is to add the following line to one of your functional test files

SimpleCov.command_name 'test:functionals' if ENV['COVERAGE']

and this line to one of the units

SimpleCov.command_name 'test:units' if ENV['COVERAGE']

Leave off the if ENV['COVERAGE'] if you're not using that variable.

Christoph Olszowka colszowka reopened this
Christoph Olszowka colszowka referenced this issue from a commit
Christoph Olszowka Prevent libraries that tamper with ARGV (I'm looking at you, rake)
from disabling SimpleCov's ability to guess test suite / framework names
based upon the command line call that invoked the tests.

Fixes #110 and probably also #45

Cleaned up and refactored the command guessing along the way.
23a768d
J. Pablo Fernández pupeno referenced this issue from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.