simple code coverage tool #5423
Conversation
| if (vec.size() <= (size_t)line) | ||
| vec.resize(line+1, NULL); | ||
| if (vec[line] == NULL) | ||
| vec[line] = new GlobalVariable(*jl_Module, T_int64, false, GlobalVariable::ExternalLinkage, |
|
Very exciting. |
|
sweet!! Doesn't seem to work in global scope yet, but seems to work fine in functions. |
|
LLVM has extremely efficient coverage support provided by the asan infrastructure. May be worth looking into. |
|
ASAN? |
|
I'm guessing it's the address sanitizer and not that. |
|
AddressSanitizer |
|
Actually, never mind. I'm seeing that was obsoleted in favor of a clang-based approach. |
|
Just to make sure I understand this, we have some sort of test script If we run |
|
This is really exciting! |
To use the tool, pass --code-coverage. On exit, for each source file foo.jl for which we have data, write foo.jl.cov next to the original file, with execution counts in front of each line, or "-" if no data. also start consolidating compiler options into a jl_compileropts struct
switch line counters to internal linkage
|
Could we do something similar to make a callgraph tool? |
simple code coverage tool
|
Great feature! I'd like to include this in automated tools like FactCheck. For this it would be great if either
|
|
+1 for redirection to a more controlled environment. Running this tool on DataFrames affects a bunch of other repos that I'd rather leave in a pristene state. |
| { "lisp", no_argument, &lisp_prompt, 1 }, | ||
| { "help", no_argument, 0, 'h' }, | ||
| { "sysimage", required_argument, 0, 'J' }, | ||
| { "code-coverage", no_argument, &jl_compileropts.code_coverage, 1 }, |
There was a problem hiding this comment.
@JeffBezanson I think this broke my Windows build;
$ cd ui
$ make VERBOSE=1 DEFAULT_REPL=basic XC_HOST=x86_64-w64-mingw32
x86_64-w64-mingw32-gcc -std=gnu99 -pipe -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -DJL_SYSTEM_IMAGE_PATH="/sys.ji" -D_WIN32_WINNT=0x0600 -O3 -falign-functions -momit-leaf-frame-pointer -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -I/home/sabae/tmp/wintest/julia/src -I/home/sabae/tmp/wintest/julia/src/support -I/home/sabae/tmp/wintest/julia/usr/include -c repl.c -o repl.o
repl.c: In function 'parse_opts':
repl.c:56:9: error: initializer element is not constant
{ "code-coverage", no_argument, &jl_compileropts.code_coverage, 1 },
^
repl.c:56:9: error: (near initialization for 'longopts[6].flag')
make: *** [repl.o] Error 1
I'm guessing this is because the cross-compiler is running in some C mode that "normal" native compiles don't? Is there a simple fix we can take to ensure this doesn't happen even in strict C modes?
|
@JeffBezanson is this something you'd want to mention in NEWS.md, or is it staying a "hidden" feature? |
|
I noticed this feature today, very neat! |
|
I knew this existed but somehow missed that it has been merged. |
|
(@IainNZ have just done an interface to Coveralls. Thanks!) |
|
One happy customer https://coveralls.io/r/loladiro/SIUnits.jl |
|
I've notice that lines of functions which are never called receive a no data dash "-" rather than a "0": being then This is leading Coverage.jl to report values to Coveralls higher than they theoretically are... Not sure if this is something to be fixed on this tool or in Coverage.jl tough. |
To use the tool, pass --code-coverage. On exit, for each source file foo.jl for which we have data, write foo.jl.cov next to the original file, with execution counts in front of each line, or "-" if no data.
Also start consolidating compiler options into a jl_compileropts struct. I think we will want to add many more options, and this will make it easier.