Hotfix: Fix regression in how test depth is calculated; add exec index #485
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a little bit of a dual part PR - first a fix to a regression introduced over the course of preparing #463, and the other piece is a small feature enhancement that enables a lot of powerful functionality.
Depth calculation fix
Between the initial PR for #463 to the final commit, I went from just recording the final test frame to instead recording the entire test call stack up until the first
covr::count
call. Unfortunately, I forgot to update the way that the stack depth of the test trace was calculated. This has been fixed now by taking the length of the stack trace, instead of (what was) the index of the frame to begin on.Unfortunately, this was silently causing incorrect depth because of
rbind
's value recycling. (effectively doing the same asrbind(c(1, 2), c(3, NULL))
) (location in diff).All is good now, and I added in some tests to avoid this from happening in the future.
Feature Addition: Test execution index
As well, I added in an index for each trace. I'm pretty excited to have this introduced. Although it's just a tiny bit of added information, it effectively means that we can reconstruct the execution path that a test took while evaluating which opens up a lot of opportunities for analyzing the test traces.
In practice we get something like this (below), where
i
records the order in which each trace is executed by each test. Combining this matrix across traces and sorting by "test" and "i" would let us see the order of coverage counts executed by a test.Materially, this means storing an extra column to the tests matrix for each trace. It comes at no substantial computational or object size cost.
?covr.record_tests
documentation to describe new data column