Description
Hello, I'm working on a fairly large Rails application utilizing SimpleCov for test coverage reporting.
We are using SimpleCov version 0.22
and Rails 6.1.7.7
.
Here is the SimpleCov configuration we are using as seen in our config/spec_helper.rb
file:
SimpleCov.start :rails do
add_filter ['/vendor/', '/lib/pmp_credentials/']
command_name "RSpec #{rand(1000000)}"
merge_timeout 3600 # increase default because suite takes long time
end
We are also using a .simplecov
configuration file whose contents are such:
class LineFilter < SimpleCov::Filter
def matches?(source_file)
source_file.lines.count < filter_argument
end
end
SimpleCov.start :rails do
use_merging true
merge_timeout 3600
add_filter '/.gem/'
add_filter '/vendor/'
add_filter '/config/'
add_filter '/lib/tasks/'
add_filter '/script/'
add_filter '/lib/*.rb'
add_group 'Services', 'app/services'
add_group 'Workers', 'app/workers'
add_group 'Trailblazer', 'app/concepts'
add_group 'Long files' do |src_file|
src_file.lines.count > 100
end
add_group "Short files", LineFilter.new(5) # Using the LineFilter class defined in Filters section above
end
SimpleCov.minimum_coverage 80 if ENV['CHECK_COVERAGE']
The problem we are experiencing is that after adding ~100
new specs and making additions to an existing FactoryBot factory, using ~1000
new lines of spec code to do so, the coverage calculation has changed dropping us below our minimum coverage enforcement by 10%
.
Here are the SImpleCov calculations before adding the new specs:
All Files ( 80.34% covered at 2594.63 hits/line )
1803 files in total.
52115 relevant lines, 41867 lines covered and 10248 lines missed. ( 80.34% )
and here are the SimpleCov calculations after adding the new specs:
All Files ( 71.91% covered at 3880.83 hits/line )
1803 files in total.
53406 relevant lines, 38406 lines covered and 15000 lines missed. ( 71.91% )
No new app
code was added in this time, only the spec
code mentioned previously. The amount of files has not changed but SimpleCov is picking up 1291 new lines of code and 3461 less lines covered with the simple addition of test code. This for us is unexpected behavior. I've searched for similar issues but was unable to find any. Any ideas off the top of the head for what the underlying issue may be? I'm willing to devote some time digging into SimpleCov's source code should the issue be an actual bug in the library but will need some guidance.
Thanks for all your hard work!!