I'm using jmockit line/path coverage to grade my students projects on unit testing, and I'm getting what looks like inconsistent results. Specifically for exit points inside if inside loops. I have stripped down a project to a minimum to show the error.
The repo can be found here:
The example is very simple. A class Portfolio contains an array of Stock. There is a method hasStock() in Portfolio to find Stock elements. I want to test that a Stock is found when the array contains 3 elements. There are 3 cases: the item is found in first, second or third place.
If the item is found in first place, the results are as expected:
If the item is found in second place, things start to get weird. First, the if is reported to be executed 3 times, when it should be 2. But the problem is that even when the return true is reported to be executed 1 time, path A is not !
And if the item is found in third place, line coverage reports that the if is executed 5 times when it should be 3, and still path A is not covered when return true is executed 1 time:
These results look like inconsistent to me. Am I missing something?
Thanks in advance
The execution count for line 33 is wrong in the second and third tests.
Thanks for reporting the bug!
And also, path A should show as covered, right?
Any news on this ? If you could give me some insight about where could be the problem, maybe I could try to find the bug and fix it.
I have a similar problem with path coverage. Any update on the fix?
No updates yet.
Faced the same issue.
Just +1 if possible to address it. Thanks
I've dug into the code and now I understand the problem. I wouldn't call it a bug but it is a case that is not dealt with. It is related to not dealing with GOTO instructions that go backwards. It generates an incorrect graph, and I cannot find and easy solution ... I've tried a couple, and it solves the above problem, but then some other coverage tests fail.
My question for @rliesenfeld and others, would it be ok if I transform the current path coverage into prime path coverage as described in the book Introduction to Software Testing ?
"would it be ok if I transform the current path coverage into prime path coverage"?
I don't really understand what would be the implications of that, but no, probably not.
I haven't made an attempt at solving this issue yet. So, for now I can't make any decision regarding the best way to proceed.
Ok. Since I need a reliable path coverage tool for my course (it has already started), I'm going to start the work in my fork and use it with my students. I'll let you know when it is finished and then you can decide.
Maybe it could be included as another coverage method instead of replacing the actual one.
I have a first implementation of prime path coverage at:
It is still very preliminary, probably buggy, but usable. I have built a github maven repository so it can be easily used just configuring the pom.xml file. Instructions in the README. Comments are welcome.