Skip to content

Loading…

Compare code coverage with saved state (like rcov's "--text-coverage-diff") #90

Closed
igal opened this Issue · 5 comments

4 participants

@igal

My favorite rcov feature is the one that displays new lines of uncovered code. This fantastic feature helps me avoid increasing technical debt by rejecting or improving patches that lack adequate tests because these are the most likely to contain errors.

It would be awesome to have this feature in simplecov.

@colszowka
Owner

That's a great idea, the code tracking code might end up being quite a mess though (i.e. differentiating between changed/new lines etc). I'll have to give it some thought and maybe also look into how rcov accomplishes this :)

@aksrikanth

I'd like to add my vote for this feature. It would be very nice to have this. Perhaps the simplest way would be to compare the output JSON for two runs to generate a "diff" view.

@pmenglund

I was just looking at how to implement coverage trending, and it will require saving the state for previous runs. Either .resultset.json can be changed to contain an array of results, or a timestamped file be generated for each coverage run.

Do you have any preference of which?

@colszowka
Owner

@pmenglund I think having separate files would be better, as it would be more obvious where to find what. One thing that has to be handled in some way are complete vs. selective test runs: If I run my test suite completely and have that stored as a resultset, then just fire up one individual suite or a single unit test file even, the report shouldn't complain about that, but rather in some way detect what has happend. I think by combining the test suite list of a resultset with some reasonable change metrics (probably coverage will never drop by more than 5% at once) we could get some solid structure here.

@infertux infertux referenced this issue
Commit has since been removed from the repository and is no longer available.
@infertux infertux referenced this issue
Commit has since been removed from the repository and is no longer available.
@infertux infertux added a commit to infertux/simplecov that referenced this issue
@infertux infertux Support for minimal coverage and maximal coverage drop
Fixes #96
References #90
6ce1097
@colszowka
Owner

Thanks to @infertux this is now implemented as the maximum_coverage_drop config option in master. Release coming in the next few days.

@colszowka colszowka closed this
@jperkin jperkin pushed a commit to joyent/pkgsrc that referenced this issue
taca Update ruby-simplecov.
Make this package to Ruby 1.9.3 only.

v0.7.1, 2012-10-12 ([changes](colszowka/simplecov@v0.7.0...v0.7.1))
-------------------

  * [BUGFIX] The gem packages of 0.7.0 (both simplecov and simplecov-html) pushed to Rubygems had some file
    permission issues, leading to problems when installing SimpleCov in a root/system Rubygems install and then
    trying to use it as a normal user (see colszowka/simplecov#171, thanks @envygeeks
    for bringing it up). The gem build process has been changed to always enforce proper permissions before packaging
    to avoid this issue in the future.


v0.7.0, 2012-10-10 ([changes](colszowka/simplecov@v0.6.4...v0.7.0))
-------------------

  * [FEATURE] The new `maximum_coverage_drop` and `minimum_coverage` now allow you to fail your build when the
    coverage dropped by more than what you allowed or is below a minimum value required. Also, `refuse_coverage_drop` disallows
    any coverage drops between test runs.
    See colszowka/simplecov#151, colszowka/simplecov#11,
    colszowka/simplecov#90, and colszowka/simplecov#96 (thanks to @infertux)
  * [FEATURE] SimpleCov now ships with a built-in MultiFormatter which allows the easy usage of multiple result formatters at
    the same time without the need to write custom wrapper code.
    See colszowka/simplecov#158 (thanks to @nikitug)
  * [BUGFIX] The usage of digits, hyphens and underscores in group names could lead to broken tab navigation
    in the default simplecov-html reports. See colszowka/simplecov-html#14 (thanks to @ebelgarts)
  * [REFACTORING] A few more ruby warnings removed. See colszowka/simplecov#106 and
    colszowka/simplecov#139. (thanks to @lukejahnke)
  * A [Pledgie button](colszowka/simplecov@63cfa99) for those that
    feel generous :)
  * The usual bunch of README fixes and documentation tweaks. Thanks to everyone who contributed those!
9027cf8
@jsonn jsonn pushed a commit to jsonn/pkgsrc that referenced this issue
taca Update ruby-simplecov.
Make this package to Ruby 1.9.3 only.

v0.7.1, 2012-10-12 ([changes](colszowka/simplecov@v0.7.0...v0.7.1))
-------------------

  * [BUGFIX] The gem packages of 0.7.0 (both simplecov and simplecov-html) pushed to Rubygems had some file
    permission issues, leading to problems when installing SimpleCov in a root/system Rubygems install and then
    trying to use it as a normal user (see colszowka/simplecov#171, thanks @envygeeks
    for bringing it up). The gem build process has been changed to always enforce proper permissions before packaging
    to avoid this issue in the future.


v0.7.0, 2012-10-10 ([changes](colszowka/simplecov@v0.6.4...v0.7.0))
-------------------

  * [FEATURE] The new `maximum_coverage_drop` and `minimum_coverage` now allow you to fail your build when the
    coverage dropped by more than what you allowed or is below a minimum value required. Also, `refuse_coverage_drop` disallows
    any coverage drops between test runs.
    See colszowka/simplecov#151, colszowka/simplecov#11,
    colszowka/simplecov#90, and colszowka/simplecov#96 (thanks to @infertux)
  * [FEATURE] SimpleCov now ships with a built-in MultiFormatter which allows the easy usage of multiple result formatters at
    the same time without the need to write custom wrapper code.
    See colszowka/simplecov#158 (thanks to @nikitug)
  * [BUGFIX] The usage of digits, hyphens and underscores in group names could lead to broken tab navigation
    in the default simplecov-html reports. See colszowka/simplecov-html#14 (thanks to @ebelgarts)
  * [REFACTORING] A few more ruby warnings removed. See colszowka/simplecov#106 and
    colszowka/simplecov#139. (thanks to @lukejahnke)
  * A [Pledgie button](colszowka/simplecov@63cfa99) for those that
    feel generous :)
  * The usual bunch of README fixes and documentation tweaks. Thanks to everyone who contributed those!
4ceab0f
@jsonn jsonn pushed a commit to jsonn/pkgsrc that referenced this issue
taca Update ruby-simplecov.
Make this package to Ruby 1.9.3 only.

v0.7.1, 2012-10-12 ([changes](colszowka/simplecov@v0.7.0...v0.7.1))
-------------------

  * [BUGFIX] The gem packages of 0.7.0 (both simplecov and simplecov-html) pushed to Rubygems had some file
    permission issues, leading to problems when installing SimpleCov in a root/system Rubygems install and then
    trying to use it as a normal user (see colszowka/simplecov#171, thanks @envygeeks
    for bringing it up). The gem build process has been changed to always enforce proper permissions before packaging
    to avoid this issue in the future.


v0.7.0, 2012-10-10 ([changes](colszowka/simplecov@v0.6.4...v0.7.0))
-------------------

  * [FEATURE] The new `maximum_coverage_drop` and `minimum_coverage` now allow you to fail your build when the
    coverage dropped by more than what you allowed or is below a minimum value required. Also, `refuse_coverage_drop` disallows
    any coverage drops between test runs.
    See colszowka/simplecov#151, colszowka/simplecov#11,
    colszowka/simplecov#90, and colszowka/simplecov#96 (thanks to @infertux)
  * [FEATURE] SimpleCov now ships with a built-in MultiFormatter which allows the easy usage of multiple result formatters at
    the same time without the need to write custom wrapper code.
    See colszowka/simplecov#158 (thanks to @nikitug)
  * [BUGFIX] The usage of digits, hyphens and underscores in group names could lead to broken tab navigation
    in the default simplecov-html reports. See colszowka/simplecov-html#14 (thanks to @ebelgarts)
  * [REFACTORING] A few more ruby warnings removed. See colszowka/simplecov#106 and
    colszowka/simplecov#139. (thanks to @lukejahnke)
  * A [Pledgie button](colszowka/simplecov@63cfa99) for those that
    feel generous :)
  * The usual bunch of README fixes and documentation tweaks. Thanks to everyone who contributed those!
8758c51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.