integer-overflow

Description

This version of JaCoCo adds the functionality to track line execution count. Essentially, we have changed the underlying data structure from a boolean[] to an int[] and increment values on every execution. This implementation is derived from JaCoCo version v0.8.7.

Implementation details

We have changed the bytecode injected in instrumented classes to hold an int[] instead of a boolean[] and then for its probes inject the following bytecode instead of that described in JaCoCo documentation:

ALOAD     <array>
xPUSH     <probeid>
DUP2
IALOAD
ICONST_1
IADD
IASTORE

The Java equivalent of this code is: array[index]++ instead of array[index] = true. Next to the added bytecode instructions, the maximum stack size has increased from 3 to 4.

The execution counts are reported only on source code lines (see caveats below) in XML and HTML reporting. The XML reporting exposes a new attribute called ec (execution count) and the HTML reporting shows the execution count of a line once hovered.

Performance

For our performance benchmarking, we have performed 20 local Maven builds running tests of a different application with the Maven goals jacoco:prepare-agent and jacoco:report executed. We compared performance against version v0.8.7 (which this code is based upon). These benchmarks have shown, using a Tukey-Kramer test after validating its assumptions, that we are 95% confident performance has decreased between ~0.9% and ~2.3%. However, this performance decrease might be different on other projects or in other environments.

Note that we can not state that this version significantly differs in performance from that of long-overflow.

Caveats

We made some decisions in our implementation that came with some caveats. These caveats are described below.

• This implementation is suspectible to integer overflowing. Therefore, when code is executed more than Integer.MAX_VALUE times, it will report negative execution counts (and therefore consider lines uncovered) or once it passes 0, it will document wrong execution counts. This can be mitigated by dumping data often and resetting the probe array.
• As JaCoCo maps instruction execution to source code lines, we can not determine on a statement level how often a statement has been executed. The heuristic that we chose to implement it to take the maximum number of executions of all instruction on a single line and report that. We mitigate this threat to validity based on the fact that most code is formatted such that statements are spread out over multiple lines.
• The implementation of the injected bytecode that updates the execution data is not thread safe. Synthetic tests show that with such an implementation, 12 threads on a 12 virtual core system misses more than half of the executions when they all update a single value in a array. For our situation, we consider this threat mitigated as this will likely not occur as much in a real scenario (different start times, different execution paths) nor will it impact the resulting analysis too much if some executions were not tracked. However, for your situation this might differ.
• Testing has changed to a minimal level to make the build pass. Therefore, most of them are based on executions counts of 0 or 1 (executed or not). Some tests have been altered to handle larger execution counts (e.g. when merging instructions). However, this test coverage is not high enough to have the functionality tested with confidence.
• As this fork is to be used in the dynamic analysis of a project as part of a master thesis, we have applied some changes that are specific to this thesis. The biggest change is that we do not ignore generated classes by org.immutables.value.Generated and javax/annotation/processing/Generated.
• This has been implemented for a specific scenario. Please use at your own risk (as enforced by license).