-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
2 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a973c36
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the title, I wonder if you were surprised at the lack of shame from the reported number. You probably know this already, but the coverage fraction reported by Coveralls is not yet all that useful. JuliaLang/julia#7541. For example, https://coveralls.io/builds/1135329 shows that for some files only 10% of the file was deemed "relevant."
Still, this is a good addition! Someday the coverage numbers will get better. And as long as a human interprets the result,
code-coverage
is still very useful for determining how good the test coverage is.a973c36
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was pleasantly surprised. I imagine it would have been much lower a few weeks ago, before you added tests for colorspace conversions.
colorspaces.jl is mostly type declarations, and should have few relevant lines, but the results for differences.jl is surprising. Is a line irrelevant if it's in a function that's never called? That seems pretty weird.
a973c36
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, it just means the function was never compiled (because it was never called). The way
code_coverage
works is that the compiler inserts the equivalent ofright before each statement, and when it starts to compile a function it first initializes all those entries to 0. So the ones deemed "relevant" are the ones that have a number in front of them in the .cov files, and so the only ones that count against you as far as the badge is concerned are the ones that stayed 0. Hence, the 7% that we lost basically reflects branches inside functions that aren't covered by current tests. Mostly the numbers are an underestimate (sometimes, a gross underestimate), but inlining can also cause it to go the other way: the compiler doesn't "credit" the source line when it gets inlined.
Bottom line: the badge is nowhere close to reality. My conversion tests might have actually decreased the stated coverage.
Right now code_coverage is useful with human interpretation; you can easily scan the differences.jl file and see what tests we're missing.