Always 100% test coverage with Ruby 2.0p353 and Rails 4.1beta

[clupprich]

I'm running the latest official patch version of Ruby 2.0 (p353) together with the the revision e4b9f6c of simplecov on Rails 4.1beta. I've added simplecov to the top of spec_helper but when I run bin/rake spec, it always reports 100% test coverage and doesn't see files with no test coverage (like certain controllers, etc.). Anyone experiencing the same problems?

I tried adapting the Spork logic to Spring, but without luck.

[clupprich]

Hm, just tried running without Spring (via rake spec) and it still reports 100% test coverage and doesn't see any controllers.

[andruby]

I seem to have the same problem with regular tests with rails 4.1.0.beta1 both with and without spring.
Simplecov loading in test_helper.rb:

require 'simplecov'
SimpleCov.start 'rails' do
  add_group "Serializers", "app/serializers"
end

[mscharley]

I'm not using rails, but had a similar issue #277 and fixed it by rolling back to ruby 1.9.3. Try that if possible.

[pmontrasio]

As a side note, no problems with 2.1.0 x86_64, rvm 1.25.14, bundler 1.5.2, rails 4.0.2 on Linux.
I added the gem one hour ago to the Gemfile of a project.

gem 'simplecov', :require => false, :group => :test

bundle install
bundle exec rake spec

and it works.

$ bundle list |grep rspec
  * rspec-core (2.14.7)
  * rspec-expectations (2.14.4)
  * rspec-mocks (2.14.4)
  * rspec-rails (2.14.1)
$ bundle list |grep simplecov
  * simplecov (0.8.2)
  * simplecov-html (0.8.0)

I'm not using spring.

[clupprich]

True, Ruby 2.1 works without a problem. Okay, I'll take that ;)

[bf4]

Spring and SimpleCov don't mix well. Your Rakefile may also be loading library code. Also see my comment in #314 (comment)

[swrobel]

Make Spring & SimpleCov play nice

[bf4]

@swrobel PR to add this to the README?

[swrobel]

#341

[swrobel]

Upon further testing of my solution, I've found that using simplecov with spring causes lower-than-actual coverage to be reported. I'm still stuck without a good solution to make the two work well together.

If it helps anyone knowledgeable with digging into the problem, I seem to get lower coverage for models when running simplecov through spring, but not for controllers.

With Spring:

COVERAGE:  77.78% -- 231/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+---------------------------------------------+
| coverage | file                                       | lines | missed | missing                                     |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+
|   4.35%  | app/models/company.rb                      | 23    | 22     | 1-2, 4, 6-7, 9-14, 16, 20-21, 24-25, 28-... |
|   7.69%  | app/models/email.rb                        | 13    | 12     | 1-2, 4-5, 7-11, 14, 16, 18                  |
|  43.48%  | app/models/user.rb                         | 46    | 26     | 1-3, 7-8, 10, 12-13, 15-21, 23, 27, 29, ... |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                                          |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38                                  |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                                          |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                                          |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+

Without Spring:

COVERAGE:  96.30% -- 286/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+--------------------+
| coverage | file                                       | lines | missed | missing            |
+----------+--------------------------------------------+-------+--------+--------------------+
|  78.26%  | app/models/company.rb                      | 23    | 5      | 21, 25, 29, 33, 37 |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                 |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38         |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                 |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                 |
+----------+--------------------------------------------+-------+--------+--------------------+

[chrishough]

@swrobel I noticed SimpleCov now runs on every spring based rspec run. Is there a way to disable it or only run it if an ENV is set?

[sunnyrjuneja]

@swrobel I have this same problem.

[sunnyrjuneja]

@swrobel I've mounted Grape into my rails project and it is also displaying below correct coverage. See issue #363. In the next week or two, I'll start reading through the Spring source code to see if there's any reason that Spring is not correctly reporting coverage for models and rack mounted apps.

[bf4]

A preloader that forks may allow you to make the Coverage module
behave as if it could clear tracked code. You should look at the
coverage.c code

[sunnyrjuneja]

@bf4 That's helpful to know. Thanks.

[bf4]

@chrishough If you wrap SimpleCov.start in a conditional, it will conditionally run. A common one people use is if ENV["COVERAGE"] =~ /\Atrue\z/i

[sunnyrjuneja]

You can also only execute simplecov outside of Spring like this:

unless defined?(Spring)
  SimpleCov.start 'rails' do
  # ...
  end
end

[bf4]

@whatasunnyday Here's a protip for that pattern

SimpleCov.start 'rails' do
  # other config
end unless defined?(Spring)

[sunnyrjuneja]

@bf4 I've never seen that before 👍 . Thanks!

[chrishough]

@bf4 @whatasunnyday the only way I was able to get this to work correctly with spring was to create the file config/spring.rb with the following in it:

if ENV['COVERAGE']
  require 'simplecov'
  SimpleCov.start do
    coverage_dir 'tmp/coverage'
    add_filter '/.bundle/'
    add_filter '/spec/'
    add_filter '/config/'
    add_group 'Models', 'app/models'
    add_group 'Controllers', 'app/controllers'
    add_group 'Services', 'app/services'
    add_group 'Utilities', 'app/utilities'
    add_group 'Helpers', 'app/helpers'
    add_group 'Libraries', 'lib'
    add_group 'Mailers', 'app/mailers'
    add_group "Long Files" do |src_file|
      src_file.lines.count > 300
    end
    add_group 'Ignored Code' do |src_file|
      File.readlines(src_file.filename).grep(/:nocov:/).any?
    end
  end
  SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter[
    SimpleCov::Formatter::HTMLFormatter
  ]
  SimpleCov.minimum_coverage 95
end

[bf4]

@chrishough tl;dr you used an ENV var :) BTW, I recommend you move the simplecov config to a .simpelcov file in your app root.

[lolalk]

@swrobel, adding SimpleCov.command_name 'Rspec' fixed the lower coverage issue, the file would be :

if ENV['RAILS_ENV'] == 'test'
  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.command_name 'Rspec'
end

Simplecov tries to discover the command but the spring command makes it impossible.
See these : http://stackoverflow.com/questions/17255475/making-spring-gem-work-with-simplecov.
And these : https://github.com/colszowka/simplecov#test-suite-names.