New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always 100% test coverage with Ruby 2.0p353 and Rails 4.1beta #271

Closed
clupprich opened this Issue Dec 21, 2013 · 22 comments

Comments

Projects
None yet
9 participants
@clupprich

clupprich commented Dec 21, 2013

I'm running the latest official patch version of Ruby 2.0 (p353) together with the the revision e4b9f6c of simplecov on Rails 4.1beta. I've added simplecov to the top of spec_helper but when I run bin/rake spec, it always reports 100% test coverage and doesn't see files with no test coverage (like certain controllers, etc.). Anyone experiencing the same problems?

I tried adapting the Spork logic to Spring, but without luck.

@clupprich

This comment has been minimized.

clupprich commented Dec 21, 2013

Hm, just tried running without Spring (via rake spec) and it still reports 100% test coverage and doesn't see any controllers.

@andruby

This comment has been minimized.

andruby commented Jan 3, 2014

I seem to have the same problem with regular tests with rails 4.1.0.beta1 both with and without spring.
Simplecov loading in test_helper.rb:

require 'simplecov'
SimpleCov.start 'rails' do
  add_group "Serializers", "app/serializers"
end
@mscharley

This comment has been minimized.

mscharley commented Jan 19, 2014

I'm not using rails, but had a similar issue #277 and fixed it by rolling back to ruby 1.9.3. Try that if possible.

@pmontrasio

This comment has been minimized.

pmontrasio commented Jan 19, 2014

As a side note, no problems with 2.1.0 x86_64, rvm 1.25.14, bundler 1.5.2, rails 4.0.2 on Linux.
I added the gem one hour ago to the Gemfile of a project.

gem 'simplecov', :require => false, :group => :test

bundle install
bundle exec rake spec

and it works.

$ bundle list |grep rspec
  * rspec-core (2.14.7)
  * rspec-expectations (2.14.4)
  * rspec-mocks (2.14.4)
  * rspec-rails (2.14.1)
$ bundle list |grep simplecov
  * simplecov (0.8.2)
  * simplecov-html (0.8.0)

I'm not using spring.

@clupprich

This comment has been minimized.

clupprich commented Jan 19, 2014

True, Ruby 2.1 works without a problem. Okay, I'll take that ;)

@bf4

This comment has been minimized.

Collaborator

bf4 commented Jul 13, 2014

Spring and SimpleCov don't mix well. Your Rakefile may also be loading library code. Also see my comment in #314 (comment)

@bf4 bf4 closed this Jul 13, 2014

@swrobel

This comment has been minimized.

Contributor

swrobel commented Oct 24, 2014

@bf4

This comment has been minimized.

Collaborator

bf4 commented Oct 24, 2014

@swrobel PR to add this to the README?

@swrobel

This comment has been minimized.

Contributor

swrobel commented Oct 24, 2014

@swrobel

This comment has been minimized.

Contributor

swrobel commented Dec 2, 2014

Upon further testing of my solution, I've found that using simplecov with spring causes lower-than-actual coverage to be reported. I'm still stuck without a good solution to make the two work well together.

If it helps anyone knowledgeable with digging into the problem, I seem to get lower coverage for models when running simplecov through spring, but not for controllers.

With Spring:

COVERAGE:  77.78% -- 231/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+---------------------------------------------+
| coverage | file                                       | lines | missed | missing                                     |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+
|   4.35%  | app/models/company.rb                      | 23    | 22     | 1-2, 4, 6-7, 9-14, 16, 20-21, 24-25, 28-... |
|   7.69%  | app/models/email.rb                        | 13    | 12     | 1-2, 4-5, 7-11, 14, 16, 18                  |
|  43.48%  | app/models/user.rb                         | 46    | 26     | 1-3, 7-8, 10, 12-13, 15-21, 23, 27, 29, ... |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                                          |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38                                  |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                                          |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                                          |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+

Without Spring:

COVERAGE:  96.30% -- 286/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+--------------------+
| coverage | file                                       | lines | missed | missing            |
+----------+--------------------------------------------+-------+--------+--------------------+
|  78.26%  | app/models/company.rb                      | 23    | 5      | 21, 25, 29, 33, 37 |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                 |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38         |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                 |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                 |
+----------+--------------------------------------------+-------+--------+--------------------+
@chrishough

This comment has been minimized.

chrishough commented Dec 12, 2014

@swrobel I noticed SimpleCov now runs on every spring based rspec run. Is there a way to disable it or only run it if an ENV is set?

@sunnyrjuneja

This comment has been minimized.

sunnyrjuneja commented Jan 29, 2015

@swrobel I have this same problem.

@sunnyrjuneja

This comment has been minimized.

sunnyrjuneja commented Jan 29, 2015

@swrobel I've mounted Grape into my rails project and it is also displaying below correct coverage. See issue #363. In the next week or two, I'll start reading through the Spring source code to see if there's any reason that Spring is not correctly reporting coverage for models and rack mounted apps.

@bf4

This comment has been minimized.

Collaborator

bf4 commented Jan 30, 2015

A preloader that forks may allow you to make the Coverage module
behave as if it could clear tracked code. You should look at the
coverage.c code

@sunnyrjuneja

This comment has been minimized.

sunnyrjuneja commented Jan 30, 2015

@bf4 That's helpful to know. Thanks.

@bf4

This comment has been minimized.

Collaborator

bf4 commented Jan 30, 2015

@chrishough If you wrap SimpleCov.start in a conditional, it will conditionally run. A common one people use is if ENV["COVERAGE"] =~ /\Atrue\z/i

@sunnyrjuneja

This comment has been minimized.

sunnyrjuneja commented Jan 30, 2015

You can also only execute simplecov outside of Spring like this:

unless defined?(Spring)
  SimpleCov.start 'rails' do
  # ...
  end
end
@bf4

This comment has been minimized.

Collaborator

bf4 commented Jan 31, 2015

@whatasunnyday Here's a protip for that pattern

SimpleCov.start 'rails' do
  # other config
end unless defined?(Spring)
@sunnyrjuneja

This comment has been minimized.

sunnyrjuneja commented Jan 31, 2015

@bf4 I've never seen that before 👍 . Thanks!

@chrishough

This comment has been minimized.

chrishough commented Feb 1, 2015

@bf4 @whatasunnyday the only way I was able to get this to work correctly with spring was to create the file config/spring.rb with the following in it:

if ENV['COVERAGE']
  require 'simplecov'
  SimpleCov.start do
    coverage_dir 'tmp/coverage'
    add_filter '/.bundle/'
    add_filter '/spec/'
    add_filter '/config/'
    add_group 'Models', 'app/models'
    add_group 'Controllers', 'app/controllers'
    add_group 'Services', 'app/services'
    add_group 'Utilities', 'app/utilities'
    add_group 'Helpers', 'app/helpers'
    add_group 'Libraries', 'lib'
    add_group 'Mailers', 'app/mailers'
    add_group "Long Files" do |src_file|
      src_file.lines.count > 300
    end
    add_group 'Ignored Code' do |src_file|
      File.readlines(src_file.filename).grep(/:nocov:/).any?
    end
  end
  SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter[
    SimpleCov::Formatter::HTMLFormatter
  ]
  SimpleCov.minimum_coverage 95
end
@bf4

This comment has been minimized.

Collaborator

bf4 commented Feb 2, 2015

@chrishough tl;dr you used an ENV var :) BTW, I recommend you move the simplecov config to a .simpelcov file in your app root.

@lolalk

This comment has been minimized.

lolalk commented Apr 8, 2015

@swrobel, adding SimpleCov.command_name 'Rspec' fixed the lower coverage issue, the file would be :

if ENV['RAILS_ENV'] == 'test'
  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.command_name 'Rspec'
end

Simplecov tries to discover the command but the spring command makes it impossible.
See these : http://stackoverflow.com/questions/17255475/making-spring-gem-work-with-simplecov.
And these : https://github.com/colszowka/simplecov#test-suite-names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment