forked from jscruggs/metric_fu
-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix simplecov rcov calculation #226
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JRuby failed with
|
::MetricFu::Calculate.integer_percent(numerator, denominator) | ||
end | ||
|
||
def self.percent_run(lines) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be refactored, perhaps much of it go into ::MetricFu::Calculate
. It's certainly pretty noisy code in here.
Otherwise, metric_fu code will be loaded before SimpleCov turns Coverage on and it will not be tracked
Will break old RCov behavior where a file is that is loaded, but no lines executed, will have a coverage percentage. Under the new rubric, it will be 0.0, which really is more correct. Since we've already removed support for running via RCov, this won't be a problem.
bf4
added a commit
that referenced
this pull request
May 2, 2014
see #226 jruby/jruby#1196 https://jira.codehaus.org/browse/JRUBY-6106 simplecov-ruby/simplecov#86 Travis ENV modified per http://docs.travis-ci.com/user/ci-environment/#Environment-variables @enebo https://twitter.com/tom_enebo/status/462289024107814914 recommended JRUBY_OPTS="-Xcli.debug=true" but that didn't work and caused other failures JRUBY_OPTS="--debug" Still produced the below error, as did JRUBY_OPTS="--debug -Xcli.debug=true" Failure message was home/travis/build/metricfu/metric_fu/gemfiles/vendor/bundle/jruby/1.9/gems/simplecov-0.8.2/lib/simplecov.rb:31 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag SimpleCov::Formatter::MetricFu calculates the same coverage from an RCov report as from SimpleCov Failure/Error: expect(source_file.coverage.count).to eq(covered_lines_from_rcov_text.count) expected: 11 got: 8 (compared using ==) # /home/travis/build/metricfu/metric_fu/spec/metric_fu/metrics/rcov/simplecov_formatter_spec.rb:52:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)'
see #226 jruby/jruby#1196 https://jira.codehaus.org/browse/JRUBY-6106 simplecov-ruby/simplecov#86 Travis ENV modified per http://docs.travis-ci.com/user/ci-environment/#Environment-variables @enebo https://twitter.com/tom_enebo/status/462289024107814914 recommended JRUBY_OPTS="-Xcli.debug=true" but that didn't work and caused other failures JRUBY_OPTS="--debug" Still produced the below error, as did JRUBY_OPTS="--debug -Xcli.debug=true" Failure message was home/travis/build/metricfu/metric_fu/gemfiles/vendor/bundle/jruby/1.9/gems/simplecov-0.8.2/lib/simplecov.rb:31 warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag SimpleCov::Formatter::MetricFu calculates the same coverage from an RCov report as from SimpleCov Failure/Error: expect(source_file.coverage.count).to eq(covered_lines_from_rcov_text.count) expected: 11 got: 8 (compared using ==) # /home/travis/build/metricfu/metric_fu/spec/metric_fu/metrics/rcov/simplecov_formatter_spec.rb:52:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)'
And we only turned that on for Coverage which 1) Didn't fix 87ffbac2162e0142fc4 2) We're not testing anymore for JRuby https://travis-ci.org/metricfu/metric_fu/jobs/24305005 1) MetricFu::ReekGenerator emit includes config file pattern into reek parameters when specified Failure/Error: reek.emit (anonymous error class): execution expired # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:35:in ``' # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:49:in ``' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:126:in `reek_version' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:118:in `turn_off_color' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:98:in `cli_options' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:16:in `emit' # /home/travis/build/metricfu/metric_fu/spec/metric_fu/metrics/reek/generator_spec.rb:17:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 2) MetricFu::ReekGenerator emit disables lines numbers from reek output Failure/Error: reek.emit (anonymous error class): execution expired # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:35:in ``' # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:49:in ``' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:126:in `reek_version' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:118:in `turn_off_color' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:98:in `cli_options' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:16:in `emit' # /home/travis/build/metricfu/metric_fu/spec/metric_fu/metrics/reek/generator_spec.rb:34:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 3) MetricFu::ReekGenerator emit includes files to analyze into reek parameters Failure/Error: reek.emit (anonymous error class): execution expired # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:35:in ``' # file:/home/travis/.rvm/rubies/jruby-1.7.11-d19/lib/jruby.jar!/jruby/kernel/jruby/process_manager.rb:49:in ``' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:126:in `reek_version' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:118:in `turn_off_color' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:98:in `cli_options' # /home/travis/build/metricfu/metric_fu/lib/metric_fu/metrics/reek/generator.rb:16:in `emit' # /home/travis/build/metricfu/metric_fu/spec/metric_fu/metrics/reek/generator_spec.rb:39:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 4) MetricFu given configured metrics, when run creates a data yaml file Failure/Error: expect { metric_fu }.to create_file("#{data_directory}/#{MetricFu.report_id}.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/_data/20140502.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:56:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 5) MetricFu given configured metrics, when run creates a report html file Failure/Error: expect { metric_fu }.to create_file("#{output_directory}/index.html") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/output/index.html" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:60:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 6) MetricFu given configured metrics, when run creates a report yaml file Failure/Error: expect { metric_fu }.to create_file("#{base_directory}/report.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/report.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:52:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 7) MetricFu given configured metrics, when run with command line specified out only outputs to the specified path Failure/Error: expect { metric_fu "--out customdir --no-open"}.to create_file("#{base_directory}/customdir/index.html") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/customdir/index.html" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:147:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 8) MetricFu given configured metrics, when run with configured and command line formatter outputs using command line formatter Failure/Error: expect { metric_fu "--format yaml"}.to create_file("#{base_directory}/report.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/report.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:104:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 9) MetricFu given configured metrics, when run with command line formatter outputs using command line formatter Failure/Error: expect { metric_fu "--format yaml"}.to create_file("#{base_directory}/report.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/report.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:86:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 10) MetricFu given configured metrics, when run with command line specified formatter + out outputs to the specified path Failure/Error: expect { metric_fu "--format yaml --out customreport.yml"}.to create_file("#{base_directory}/customreport.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/customreport.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:136:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 11) MetricFu given configured metrics, when run with configured formatter outputs using configured formatter Failure/Error: }.to create_file("#{base_directory}/report.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/report.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:70:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)' 12) MetricFu given configured metrics, when run with configured specified out outputs using configured out Failure/Error: }.to create_file("#{base_directory}/customreport.yml") The file "/home/travis/build/metricfu/metric_fu/tmp/metric_fu/test/customreport.yml" was not created # /home/travis/build/metricfu/metric_fu/spec/metric_fu/run_spec.rb:120:in `(root)' # /home/travis/build/metricfu/metric_fu/spec/support/timeout.rb:6:in `(root)'
bf4
added a commit
that referenced
this pull request
May 4, 2014
[Feature/Fix] Fix simplecov rcov calculation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Turns out #222 didn't really fix #153
Basically,
The RCov formatter, being faithful to the RCov format, didn't annotate 'missed' lines, which #222 did not address, and hence #153 was not fixed.
In this PR, I introduce a syntax '--' to the RCov report generator that we then parse as a 'missed' line. And all is good in the world. Now, our RCov output is no longer really RCov, and in fact true RCov docs may be processed incorrectly as RCov covered lines were identified by the absence of '!!'. See logic in RCovFormatCoverage#assemble_files
All of this leads to the need to make a stronger break from pretending to use RCov, and to, at least
"#{MetricFu.report_id}_coverage.json"
or some-such thatThat is, to no longer output an 'rcov.txt' to 'coverage/rcov/rcov.txt'
I'd love to change the metric namespace to TestCoverage.. but that's more work to do without breaking the existing contracted behavior.
It also turns out that the JRuby Coverage module produces different results from MRI. See