Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meaning of multiple X and V next to if statements #283

Closed
deblauwetom opened this issue Oct 6, 2018 · 3 comments

Comments

@deblauwetom
Copy link

@deblauwetom deblauwetom commented Oct 6, 2018

Hi,

I installed gcovr and it works very well, really easy to use(and fast in comparison with lcov !).

Now I have a question that I did not find in the documentation.

When I see an "if()" statement, it sometimes contains multiple X and V's but it is only a simple statement for example if (isValid(obj)) {. So after some searching I understood that some other people using lcov also had this problem and that it is due to some exception handling that could happen in that if-statement. Then I found this code from a guy who filters lcov files just before they generate html output to "clean up" the if statements. See: https://github.com/gsauthof/imapdl/blob/master/ci/filterbr.py
I don't know if maybe something similar could be added to gcovr? That would be cool! Or maybe just that it is clear what checkmarks are about the total if-statement itself, and what is about parts of it. That would already solve the problem for me.

Best regards,
Tom,

@latk

This comment has been minimized.

Copy link
Member

@latk latk commented Oct 7, 2018

Hi @deblauwetom, thanks for these ideas!

Gcov/gcovr/lcov use assembly code level coverage data, so they show the true branch coverage in the binary. This is neither good or bad, but this leads to many compiler-generated branches being included in the coverage calculation. For some use cases, it would be wrong to hide this information. But most of the time, these invisible branches are confusing.

I started typing a lengthy response explaining why post-processing of the source code should not be done, but the TL;DR is: parsing C/C++ is complex, especially considering macros, and how compiler optimizations might affect branches. Also note that there are many C operators that branch: for, if, while, switch/case, &&, ||, ? :, plus trigraph versions, plus alternative operator names in C++. Trying to figure out which branches are real would be super difficult, so I won't do it.

Gcov actually distinguishes normal code from code that is only reachable by exceptions. Currently, gcovr uses that to ignore uncovered lines that are only exception-reachable, but doesn't ignore branches that are only taken by exceptions. The problem is, this also applies to catch clauses, leading to inconsistent reports like this:

While the branch to the catch-clause is uncovered (✗), the code in the catch clause itself is ignored (white background) because it is only exception-reachable. The corresponding gcov report is:

        3:   17:    for (auto& v : cases) {
call    0 returned 100%
call    1 returned 100%
branch  2 taken 67% (fallthrough)
branch  3 taken 33%
        -:   18:        try {
        2:   19:            sut(v);
call    0 returned 100%
branch  1 taken 100% (fallthrough)
branch  2 taken 0% (throw)
    =====:   20:        } catch (std::exception const&) {
call    0 never executed
call    1 never executed
call    2 never executed
    =====:   21:            std::cout << "Oops" << std::endl;
call    0 never executed
branch  1 never executed
branch  2 never executed
call    3 never executed
branch  4 never executed
branch  5 never executed
        -:   22:        }
        -:   23:    }

(Note that on the std::cout line, branches 2 and 5 would also be exception-only branches, but that is invisible in this report.)

I think I'll change this:

  • All uncovered lines and branches will be shown by default.
  • Add an option that ignores exception-only lines and branches, both compiler-generated and explicit catch clauses. Or only exclude throw-branches? More thought needed.
  • The docs will get an FAQ entry about coverage with exceptions.
  • The existing --exclude-unreachable-branches option (which excludes coverage from lines that don't look like they contain code) should be kept distinct, but might need a clearer name.

This will not be perfect, but the information for a perfect solution is unavailable in this case.

@deblauwetom

This comment has been minimized.

Copy link
Author

@deblauwetom deblauwetom commented Oct 8, 2018

Thanks for the comment! Just so I understand correctly:

So currently it's almost impossible to show intuitive information with the output gcov gives you because you can't know if the branches are "exception generated" branches or "if-test" branches?
So if I see next to the my if-statement XVVV, then I don't know what caused the X really, because it could as well be from an exception that I didn't test. Too bad this can't be indicated visually indeed. I've used other tools like bullseye and they really break down the if-statements and show what happened, but that sort of thing will maybe depend on gcov itself then?

latk added a commit to latk/gcovr that referenced this issue Oct 12, 2018
@latk latk closed this in 84e85f0 Oct 12, 2018
@latk

This comment has been minimized.

Copy link
Member

@latk latk commented Oct 12, 2018

Your understanding is correct, gcov does not provide enough info to attribute a given branch to a specific operator.

I've written a FAQ entry that discusses C++ branch coverage, and added a new --exclude-throw-branches option which filters out any branches that gcov declares as exception-only. Unfortunately this cannot remove all exception-related branches, only those that transition to an exception handler (such as a catch clause, or a local object with a destructor).

You can test this option in the gcovr development version (see the installation guide) or wait for the 4.2 release which I'll probably do later this month.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.