Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always 100% test coverage with Ruby 2.0p353 and Rails 4.1beta #271

Closed
clupprich opened this issue Dec 21, 2013 · 22 comments
Closed

Always 100% test coverage with Ruby 2.0p353 and Rails 4.1beta #271

clupprich opened this issue Dec 21, 2013 · 22 comments

Comments

@clupprich
Copy link

@clupprich clupprich commented Dec 21, 2013

I'm running the latest official patch version of Ruby 2.0 (p353) together with the the revision e4b9f6c of simplecov on Rails 4.1beta. I've added simplecov to the top of spec_helper but when I run bin/rake spec, it always reports 100% test coverage and doesn't see files with no test coverage (like certain controllers, etc.). Anyone experiencing the same problems?

I tried adapting the Spork logic to Spring, but without luck.

@clupprich
Copy link
Author

@clupprich clupprich commented Dec 21, 2013

Hm, just tried running without Spring (via rake spec) and it still reports 100% test coverage and doesn't see any controllers.

@andruby
Copy link

@andruby andruby commented Jan 3, 2014

I seem to have the same problem with regular tests with rails 4.1.0.beta1 both with and without spring.
Simplecov loading in test_helper.rb:

require 'simplecov'
SimpleCov.start 'rails' do
  add_group "Serializers", "app/serializers"
end
@mscharley
Copy link

@mscharley mscharley commented Jan 19, 2014

I'm not using rails, but had a similar issue #277 and fixed it by rolling back to ruby 1.9.3. Try that if possible.

@pmontrasio
Copy link

@pmontrasio pmontrasio commented Jan 19, 2014

As a side note, no problems with 2.1.0 x86_64, rvm 1.25.14, bundler 1.5.2, rails 4.0.2 on Linux.
I added the gem one hour ago to the Gemfile of a project.

gem 'simplecov', :require => false, :group => :test

bundle install
bundle exec rake spec

and it works.

$ bundle list |grep rspec
  * rspec-core (2.14.7)
  * rspec-expectations (2.14.4)
  * rspec-mocks (2.14.4)
  * rspec-rails (2.14.1)
$ bundle list |grep simplecov
  * simplecov (0.8.2)
  * simplecov-html (0.8.0)

I'm not using spring.

@clupprich
Copy link
Author

@clupprich clupprich commented Jan 19, 2014

True, Ruby 2.1 works without a problem. Okay, I'll take that ;)

@bf4
Copy link
Collaborator

@bf4 bf4 commented Jul 13, 2014

Spring and SimpleCov don't mix well. Your Rakefile may also be loading library code. Also see my comment in #314 (comment)

@bf4 bf4 closed this Jul 13, 2014
@swrobel
Copy link
Contributor

@swrobel swrobel commented Oct 24, 2014

@bf4
Copy link
Collaborator

@bf4 bf4 commented Oct 24, 2014

@swrobel PR to add this to the README?

@swrobel
Copy link
Contributor

@swrobel swrobel commented Oct 24, 2014

@swrobel
Copy link
Contributor

@swrobel swrobel commented Dec 2, 2014

Upon further testing of my solution, I've found that using simplecov with spring causes lower-than-actual coverage to be reported. I'm still stuck without a good solution to make the two work well together.

If it helps anyone knowledgeable with digging into the problem, I seem to get lower coverage for models when running simplecov through spring, but not for controllers.

With Spring:

COVERAGE:  77.78% -- 231/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+---------------------------------------------+
| coverage | file                                       | lines | missed | missing                                     |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+
|   4.35%  | app/models/company.rb                      | 23    | 22     | 1-2, 4, 6-7, 9-14, 16, 20-21, 24-25, 28-... |
|   7.69%  | app/models/email.rb                        | 13    | 12     | 1-2, 4-5, 7-11, 14, 16, 18                  |
|  43.48%  | app/models/user.rb                         | 46    | 26     | 1-3, 7-8, 10, 12-13, 15-21, 23, 27, 29, ... |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                                          |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38                                  |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                                          |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                                          |
+----------+--------------------------------------------+-------+--------+---------------------------------------------+

Without Spring:

COVERAGE:  96.30% -- 286/297 lines in 15 files

+----------+--------------------------------------------+-------+--------+--------------------+
| coverage | file                                       | lines | missed | missing            |
+----------+--------------------------------------------+-------+--------+--------------------+
|  78.26%  | app/models/company.rb                      | 23    | 5      | 21, 25, 29, 33, 37 |
|  90.00%  | app/controllers/api/v1/base_controller.rb  | 10    | 1      | 15                 |
|  90.62%  | app/controllers/api/v1/leaks_controller.rb | 32    | 3      | 13, 25, 38         |
|  94.74%  | app/controllers/dashboard_controller.rb    | 19    | 1      | 18                 |
|  98.28%  | app/controllers/users_controller.rb        | 58    | 1      | 62                 |
+----------+--------------------------------------------+-------+--------+--------------------+
@chrishough
Copy link

@chrishough chrishough commented Dec 12, 2014

@swrobel I noticed SimpleCov now runs on every spring based rspec run. Is there a way to disable it or only run it if an ENV is set?

@sunnyrjuneja
Copy link

@sunnyrjuneja sunnyrjuneja commented Jan 29, 2015

@swrobel I have this same problem.

@sunnyrjuneja
Copy link

@sunnyrjuneja sunnyrjuneja commented Jan 29, 2015

@swrobel I've mounted Grape into my rails project and it is also displaying below correct coverage. See issue #363. In the next week or two, I'll start reading through the Spring source code to see if there's any reason that Spring is not correctly reporting coverage for models and rack mounted apps.

@bf4
Copy link
Collaborator

@bf4 bf4 commented Jan 30, 2015

A preloader that forks may allow you to make the Coverage module
behave as if it could clear tracked code. You should look at the
coverage.c code

@sunnyrjuneja
Copy link

@sunnyrjuneja sunnyrjuneja commented Jan 30, 2015

@bf4 That's helpful to know. Thanks.

@bf4
Copy link
Collaborator

@bf4 bf4 commented Jan 30, 2015

@chrishough If you wrap SimpleCov.start in a conditional, it will conditionally run. A common one people use is if ENV["COVERAGE"] =~ /\Atrue\z/i

@sunnyrjuneja
Copy link

@sunnyrjuneja sunnyrjuneja commented Jan 30, 2015

You can also only execute simplecov outside of Spring like this:

unless defined?(Spring)
  SimpleCov.start 'rails' do
  # ...
  end
end
@bf4
Copy link
Collaborator

@bf4 bf4 commented Jan 31, 2015

@whatasunnyday Here's a protip for that pattern

SimpleCov.start 'rails' do
  # other config
end unless defined?(Spring)
@sunnyrjuneja
Copy link

@sunnyrjuneja sunnyrjuneja commented Jan 31, 2015

@bf4 I've never seen that before 👍 . Thanks!

@chrishough
Copy link

@chrishough chrishough commented Feb 1, 2015

@bf4 @whatasunnyday the only way I was able to get this to work correctly with spring was to create the file config/spring.rb with the following in it:

if ENV['COVERAGE']
  require 'simplecov'
  SimpleCov.start do
    coverage_dir 'tmp/coverage'
    add_filter '/.bundle/'
    add_filter '/spec/'
    add_filter '/config/'
    add_group 'Models', 'app/models'
    add_group 'Controllers', 'app/controllers'
    add_group 'Services', 'app/services'
    add_group 'Utilities', 'app/utilities'
    add_group 'Helpers', 'app/helpers'
    add_group 'Libraries', 'lib'
    add_group 'Mailers', 'app/mailers'
    add_group "Long Files" do |src_file|
      src_file.lines.count > 300
    end
    add_group 'Ignored Code' do |src_file|
      File.readlines(src_file.filename).grep(/:nocov:/).any?
    end
  end
  SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter[
    SimpleCov::Formatter::HTMLFormatter
  ]
  SimpleCov.minimum_coverage 95
end
@bf4
Copy link
Collaborator

@bf4 bf4 commented Feb 2, 2015

@chrishough tl;dr you used an ENV var :) BTW, I recommend you move the simplecov config to a .simpelcov file in your app root.

@lolalk
Copy link

@lolalk lolalk commented Apr 8, 2015

@swrobel, adding SimpleCov.command_name 'Rspec' fixed the lower coverage issue, the file would be :

if ENV['RAILS_ENV'] == 'test'
  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.command_name 'Rspec'
end

Simplecov tries to discover the command but the spring command makes it impossible.
See these : http://stackoverflow.com/questions/17255475/making-spring-gem-work-with-simplecov.
And these : https://github.com/colszowka/simplecov#test-suite-names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants