Generate gcov coverage data

[brson]

It would be nice to know what our test coverage looks like. I believe this depends on #689.

[msullivan]

Depends on #237.

[kud1ing]

Should this issue be tagged with "debuginfo"?

[brson]

Yes

[eholk]

According to a comment in #2234, this bug is unblocked now.

[brson]

This is definitely blocked again on functioning debug info.

[bblum]

revisiting for triage. this would be especially great to have in the concurrency tests, where covering all the branches is a function of scheduling rather than input values.

[bblum]

i was already here, but now i am here again

[jdm]

This would be super-interesting to dig into now, since we have quite functional debug symbols after this summer's GSOC.

[catamorphism]

Would be awesome, but not 1.0

[mletterle]

On a related note, I found this project today: https://github.com/SimonKagstrom/kcov

kcov uses the DWARF debug-info to generate coverage data, I've verified that it appears to work with test crates. It also generates cobertura xml files of the coverage data. Might be a nice stop gap for getting coverage numbers on rust itself.

[emberian]

kcov looks much nicer than gcov overall.

[pongad]

I'm not sure how relevant this is, but Go did it in a pretty interesting way: http://blog.golang.org/cover
Not sure if everyone wants to do it the way they did.

[emberian]

I was recently looking at this. llvm has built-in gcov generation passes.

On Mon, Feb 3, 2014 at 12:54 AM, Michael Darakananda <
notifications@github.com> wrote:

I'm not sure how relevant this is, but Go did it in a pretty interesting
way: http://blog.golang.org/cover
Not sure if everyone wants to do it the way they did.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/690#issuecomment-33926365
.

[huonw]

I tested kcov, it seems to (mostly) just work with -Z extra-debug-info (output for extratest).

stdtest hits runtime aborts when running under kcov (possibly threading stuff, haven't had a chance to test) and it doesn't seem to understand our debug-info perfectly (or we have slightly misaligned/mislabeled debug-info) since sometimes lines inside ifs are marked as executed while the condition is, apparently, never run.

However, all-in-all kcov seems pretty good.

[alexcrichton]

(drive-by-triage)

LLVM's gcov pass requires some runtime support in the form of compiler-rt. We're currently bundling compiler-rt with rust, but it doesn't have gcov support compiled into it. It's plausible that we could add a first-class flag to generate gcov information which would link to this special library.

[thehydroimpulse]

/cc me

[farcaller]

Tinkered with insert-gcov-profiling a bit. If you add it as a pass:

-C passes=insert-gcov-profiling

it fails to find the appropriate runtime support (llvm_gcda_start_file, llvm_gcda_emit_function, etc.). So I tried adding those from clang's compiler-rt:

-C link-args=/usr/lib/clang/3.4/lib/linux/libclang_rt.profile-x86_64.a

Fails with

undefined reference to `atexit'

Well, at that point I just linked everything to -lc as well 😄 and surprisingly that compiled and the binary works.

Now the problem is that those two binaries have different timestamps:

% hexdump -e '"%x\n"' -s8 -n4 test.gcno
9ad3f7b9
% hexdump -e '"%x\n"' -s8 -n4 test.gcda
4c4c564d

Any ideas what to do with that?

[emberian]

@farcaller do they need to have the same timestamp?

[farcaller]

Yes, that's how they check that gcda output corresponds to gcno provided by complier. It might be because of some differences in clang's libclang_rt.profile-x86_64.a and whatever llvm generates in there at compile time, I'm rebuilding rustc with matching version of compiler-rt's profile now to verify.

[farcaller]

With matching profile part of compiler-rt:

make -C $RUST_SRC/src/compiler-rt" ProjSrcRoot="$RUST_SRC/src/compiler-rt" ProjObjRoot="$RUST_SRC/x86_64-apple-darwin/rt/compiler-rt" CC="clang" AR="ar" RANLIB="ar s" CFLAGS="-Wall -Werror -g -fPIC  -Qunused-arguments" TargetTriple=x86_64-apple-darwin clang_darwin-profile_osx

you can get gcov profiling generated by

-C passes=insert-gcov-profiling -C link-args=$RUST_SRC/x86_64-apple-darwin/rt/compiler-rt/clang_darwin/profile_osx/libcompiler_rt.a

Now, there are two more problems.

The compiler can't find __morestack_allocate_stack_space symbol. Not sure if this is directly related to the gcov though.

The bigger issue is that the file is still not parseable:

% gcov lib.gcda
Function identifiers do not match: 2 != 0 (in _ZN4zinc7drivers3lcd4test11TestLCD.LCD5clearE).
Invalid .gcda File!

[farcaller]

It actually works fine on extremely simple "hello world" binary. Could it be the case that insert-gcov-profiling pass should be inserted at another time in the pipeline?

[farcaller]

FWIW, kcov actually seems to work if -C relocation-model=static is passed to rustc (as seen in hackndev/zinc#168)

[reem]

/cc me

[steveklabnik]

Triage bump: anyone know the status of this today?

[farcaller]

rustc nightly (dcaeb6a), ubuntu 14.10, x86_64.

• a binary built with rustc -g test.rs produces empty output for kcov;
• a binary built with rustc -g -C relocation-model=static test.rs fails to link:

error: linking with `cc` failed: exit code: 1
note: cc '-Wl,--as-needed' '-m64' '-L' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib' '-o' 'test' 'test.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' '-Wl,--gc-sections' '-pie' '-nodefaultlibs' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib' '-L' '/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib' '-L' '/.rust/lib/x86_64-unknown-linux-gnu' '-L' '/lib/x86_64-unknown-linux-gnu' '-Wl,--whole-archive' '-Wl,-Bstatic' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-ldl' '-lpthread' '-lrt' '-lgcc_s' '-lpthread' '-lc' '-lm' '-lcompiler-rt'
note: /usr/bin/ld: test.o: relocation R_X86_64_32S against `.rodata.str1905' can not be used when making a shared object; recompile with -fPIC
test.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

Building with rustc -g -C passes=insert-gcov-profiling -C link-args=/usr/lib/clang/3.4/lib/linux/libclang_rt.profile-x86_64.a test.rs still works but Coverstory reports:

File checksums do not match: 2561096886 != 1280071245.
(null):Invalid .gcda File!

and gcov test.gcda:

test.gcno:version '402*', prefer '408*'

gcov: out of memory allocating 7020426464 bytes after a total of 135168 bytes

So there is no way to generate coverage data for rust that I'm aware of at the moment (that's bad).

[steveklabnik]

I'm pulling a massive triage effort to get us ready for 1.0. As part of this, I'm moving stuff that's wishlist-like to the RFCs repo, as that's where major new things should get discussed/prioritized.

This issue has been moved to the RFCs repo: rust-lang/rfcs#646