Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to configure SimpleCov with Spring? #381

Closed
onebree opened this issue Mar 30, 2015 · 11 comments

Comments

@onebree
Copy link
Contributor

commented Mar 30, 2015

Rails 4.2.0, RSpec 3.1, SimpleCov 0.9.2

I have the following in my rails_helper.rb file, outside of the RSpec.configure block

require 'simplecov'
SimpleCov.start 'rails' do
  add_filter '/app/models/legacy/'
  add_filter '/app/controllers/playlists_controller_legacy.rb'
  add_filter '/app/models/login.rb'
  add_filter '/app/models/login_auth.rb'
  add_filter '/app/models/login_config.rb'
end

I have read online that some people find running spring rspec returns high or even 100% coverage. It does not seem that way for our app, but I would just like to make sure. The LOC count seems a little low (short by 1000 maybe?), although again that could be from recent refactoring I have not viewed the commits for.

And my config/spring.rb file contains nothing about SimpleCov. Would I put the above configuration in the spring file itself, or under the Spring.after_fork block?

@MasterSlowPoke

This comment has been minimized.

Copy link

commented Apr 1, 2015

I definitely have an issue with spring and simplecov:

$ bundle exec rspec
Finished in 15.98 seconds (files took 8.6 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 2372 / 7486 LOC (31.69%) covered.

$ bundle exec spring rspec
Finished in 13.9 seconds (files took 0.65848 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 18 / 27 LOC (66.67%) covered.

This is with config.eager_load = true set in the test environment, as I'd like to have all the files, even untested ones, in the coverage report. Setting this to false 'fixes' spring:

$ bundle exec rspec
Finished in 20.97 seconds (files took 11.15 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 1505 / 3530 LOC (42.63%) covered.

$ bundle exec spring rspec
Finished in 15.01 seconds (files took 0.89234 seconds to load)
87 examples, 0 failures, 5 pending

Coverage report generated for RSpec to /Users/craigsniffen/Documents/Projects/71lbs/coverage. 1188 / 2953 LOC (40.23%) covered.

The difference in LOC is troubling, however. It's also not loading all of the files as I'd want.

@onebree

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2015

Why is the LOC count in your first bundle exec rspec higher than the second?

I have the following settings in my test.rb file:

  # The test environment is used exclusively to run your application's
  # test suite.  You never need to work with it otherwise.  Remember that
  # your test database is "scratch space" for the test suite and is wiped
  # and recreated between test runs.  Don't rely on the data there!
  config.cache_classes = false

  # Disable Rails's static asset server (Apache or nginx will already do this)
  config.serve_static_files = false
  config.eager_load = false

When running rspec alone, versus with spring, I did not notice a change in LOC or coverage. As I mentioned in my original post, we may have truly deleted a couple thousand LOC.

@MasterSlowPoke

This comment has been minimized.

Copy link

commented Apr 1, 2015

I'm going to assume that it's because it's loading files that weren't loaded in the other run. I have very poor test coverage.

I found the new section on the readme for dealing with spring, but after implementing it my coverage reports now claim that all code outside of methods are not covered.

@onebree

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2015

I am going to run some tests with eager load toggled. Thank you for the advice

UPDATE: I see what you mean. Turning eager load to true makes my coverage 0/0 LOC at 100%

@bf4

This comment has been minimized.

Copy link
Collaborator

commented Apr 1, 2015

Maybe a dumb question, but have you seen #341 (in the README?)

Also, and this is just my opinion, but spring ruins everything it touches. I can't tell you how many times some bizarre error has been due to spring running old code. My bash_profile now has export DISABLE_SPRING=1 and things have been normal since. And conceptually, I don't think there's a right way to mix persisting pre-loaded code with tracking discrete runs of executed code. Maybe some of http://tenderlovemaking.com/2015/02/13/predicting-test-failues.html will make it into SimpleCov at some point which will make this feasible. Also, check out https://github.com/danmayer/coverband

@onebree

This comment has been minimized.

Copy link
Contributor Author

commented Apr 2, 2015

I did see that part of the README. My question, do I put that in my Spring.after_fork? Or completely before then? This is my my configuration now:

# config/spring.rb
Spring.after_fork do
  if Rails.env == "test"
    class ActiveRecord::Base
      mattr_accessor :shared_connection
      @@shared_connection = nil
      def self.connection
        @@shared_connection || ConnectionPool::Wrapper.new(size: 1) { retrieve_connection }
      end
    end
    ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

    RSpec.configure do |config|
      srand; config.seed = srand % 0xFFFF
    end
  end
end
@onebree

This comment has been minimized.

Copy link
Contributor Author

commented Apr 2, 2015

@bf4 As I mentioned, I do not see any strange behavior with spring right now. I just wanted to properly set it up to prevent any behavior. When I make major changes I use spring stop and start back up again. My SimpleCov config is in my rails_helper.rb, with config.eager_load = false.

I guess I should simply not fix what ain't broken?

@bf4

This comment has been minimized.

Copy link
Collaborator

commented Apr 2, 2015

@ibrahima

This comment has been minimized.

Copy link

commented Nov 28, 2017

As a related note, since this is one of the top search results for "SimpleCov eager_load", if you do want to eager load all your code, you could try what is mentioned here: https://stormconsultancy.co.uk/blog/development/ruby-on-rails/getting-accurate-code-coverage-metrics-from-simplecov-rails-project/

just run Rails.application.eager_load! manually after SimpleCov.start

I'll submit a PR for the docs if this seems like something that should be in the documentation!

scmx added a commit to scmx/simplecov that referenced this issue May 20, 2018
scmx added a commit to scmx/simplecov that referenced this issue May 20, 2018
A cleaner solution to usage with spring than copying simple cov
start code to config/spring.rb.

According to suggestion here
colszowka#381 (comment)
and here https://stormconsultancy.co.uk/blog/development/ruby-on-rails/getting-accurate-code-coverage-metrics-from-simplecov-rails-project/
@dgilperez

This comment has been minimized.

Copy link

commented Nov 2, 2018

@ibrahima thanks for the suggestion! Unfortunately, I'm seeing this error spec/spec_helper.rb:6:in <top (required)>': uninitialized constant Rails (NameError) after adding Rails.application.eager_load! to my spec_helper.rb. Should I add the line to my rails_helper.rb, down the require list instead?

@dgilperez

This comment has been minimized.

Copy link

commented Nov 2, 2018

Just to let you know, everything is working fine when I moved Rails.application.eager_load! to rails_helper.rb juts after the require "spec_helper" line. I'm also seeing improvements in the coverage, probably this fixed our problems - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.