Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Merging doesn't work with test-unit #45

Closed
Ecco opened this Issue May 22, 2011 · 14 comments

Comments

Projects
None yet
10 participants

Ecco commented May 22, 2011

Rails 3 application. Simple tests (no rspec, no cucumber).

If I add this to my gemfile :

require 'test-unit', '2.3.0'

then merging doesn't work as unit, functional, and integration test results overwrite each-other.

Thing is, I need this for the performance tests to work.

Ecco commented May 22, 2011

For some reason, it seems like ARGV is not passed to SimpleCov::CommandGuesser when using "test-unit 2.3.0"

Ecco commented May 22, 2011

Ok, I figured it out : for some weird reason, the "unit-test" gem alters ARGV !
A very simple fix is to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start, so that the gem hasn't had time to touch ARGV.

Owner

colszowka commented May 24, 2011

Oh, ok. Haven't tried test-unit 2+ yet, but if it's the way you're saying that it messes with ARGV, that seems like a pretty bad practice to me :/

It'd be great if you could swing up a little part about test/unit 2 including your workaround as a code example for the README and phase out a pull request. Thanks!

@colszowka colszowka closed this May 24, 2011

@colszowka colszowka reopened this May 24, 2011

Owner

colszowka commented May 24, 2011

Actually reopening this as a reminder to add something about this into the README

wanchic commented Jul 29, 2011

I am also having this problem

pupeno commented Aug 26, 2011

I got this issue and I spent a couple of hours trying to figure out what was going on... are my tests not running? no, they are running, is there some old coverage data somewhere else? Until I concluded the merging wasn't working, and then upon inspecting the .yml file, realizing results were not separated by test suit. Yeah, documenting this issue seems important, and fixing it if possible too so that it doesn't happen.

I am having this issue as well. If I view the HTML file after the unit tests part, the results are different then at the end, after the functionals.

Using: $ rake test

Gemfile:
gem 'test-unit', '1.2.3'

I see the fix is: "to call SimpleCov.command_name (w/o any parameter) right after SimpleCov.start"
OK, I added SimpleCov.command_name to test_helper.rb. The first 3 lines are now:
require 'simplecov'
SimpleCov.start 'rails'
SimpleCov.command_name

But I am still seeing the same symptom. How can I put the results of both units and functionals in one report? Is it possible to create a rake task that would do it?

Also, I am running test-unit 1.2.3, so I don't know if this is an issue with just test-unit 2?

Otherwise, seems to work great. Thank you!

Still seeing this issue with 0.5.0. When using rake test, the functional tests results overwrite the unit test results.

Updated to rake 0.9.2.

OK, after working on this for 12 hours, I have narrowed it down to 2 or 3 gems. However, these gems do not always cause the problem and the problem does not disappear unless all 3 are removed:
gem 'mocha'
gem 'redgreen'
gem 'test-unit', '1.2.3'

How can you save the results from a test suite with a given name, and manually merge the results of 2 test suites?

Same problem with test-unit 2.4.0

As Ecco said, it is the test-unit who is eating the ARGV content.

A fix for test-unit 2.4.0 could be:

~/Downloads/test-unit(master) $ git diff --no-prefix 
diff --git lib/test/unit/autorunner.rb lib/test/unit/autorunner.rb
index 7a31aa1..029a4b5 100644
--- lib/test/unit/autorunner.rb
+++ lib/test/unit/autorunner.rb
@@ -146,7 +146,7 @@ module Test
       def process_args(args = ARGV)
         begin
           args.unshift(*@default_arguments)
-          options.order!(args) {|arg| @to_run << arg}
+          options.order!(args.dup) {|arg| @to_run << arg}
         rescue OptionParser::ParseError => e
           puts e
           puts options

I let it here just in case it is helpful for someone, I'll try to comunicate with the test-unit maintainers but maybe I'll forget.

nikosd commented Dec 9, 2011

Thanks @fguillen for the tip. Sent a pull request to the test-unit people (test-unit/test-unit#12).

Owner

colszowka commented Jan 27, 2012

Obviously the test-unit guys merged this into Test/Unit, I suspect by release date that v2.4.3+ should have this resolved. Please re-open if there is still trouble.

Thanks a lot for taking care of this guys!

@colszowka colszowka closed this Jan 27, 2012

mdi commented Jan 30, 2012

This problem still exists w/ test-unit 2.4.5 and simplecov 0.5.4.

yeah, still broken..

The workaround is to add the following line to one of your functional test files

SimpleCov.command_name 'test:functionals' if ENV['COVERAGE']

and this line to one of the units

SimpleCov.command_name 'test:units' if ENV['COVERAGE']

Leave off the if ENV['COVERAGE'] if you're not using that variable.

@colszowka colszowka reopened this Feb 22, 2012

@colszowka colszowka added a commit that referenced this issue Feb 22, 2012

@colszowka colszowka Prevent libraries that tamper with ARGV (I'm looking at you, rake)
from disabling SimpleCov's ability to guess test suite / framework names
based upon the command line call that invoked the tests.

Fixes #110 and probably also #45

Cleaned up and refactored the command guessing along the way.
23a768d
Collaborator

bf4 commented May 17, 2015

Closing as stale. Please reopen if this is still an issue. (And take a look at #340 )

@bf4 bf4 closed this May 17, 2015

@bf4 bf4 locked and limited conversation to collaborators May 17, 2015

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.