New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Measure unit-test code coverage #298

Closed
tezeb opened this Issue Feb 24, 2017 · 6 comments

Comments

Projects
None yet
2 participants
@tezeb
Contributor

tezeb commented Feb 24, 2017

As unit tests were recently added, it would be a nice idea to also add code coverage.
It seems as travis is able to gather gcov data, which should make it relatively easy to setup.
I'm willing to set it up(at least for Linux), although I am not a travis expert.

But, as there are also alternatives, I would like to discuss it briefly.
What are your thought?

@annejan

This comment has been minimized.

Member

annejan commented Feb 24, 2017

I would love that, since on my work projects it gives the extra bit of incentive to add more tests.

Current test coverage is 0.0% since there is still only one test for a static helper.

I have tried to set up gcov and the likes a while back but couldn't get it working in the time I had available. If anyone can get anything like it going preferably with travis that would be awesome!

@annejan annejan added the discussion label Feb 24, 2017

@annejan

This comment has been minimized.

Member

annejan commented Feb 27, 2017

We should be able to use https://github.com/pcolby/lcoveralls like (for example) https://github.com/pcolby/libqtaws seems to be doing . .

@annejan

This comment has been minimized.

Member

annejan commented Feb 28, 2017

Travis is now compiling and running unit test on both Linux (gcc and clang) and macOS (gcc and clang)
https://travis-ci.org/IJHack/QtPass/builds/206105212

Appveyor is now compiling and running unit test too, and even with this one dumb test I've found a bug!
https://ci.appveyor.com/project/annejan/qtpass/build/1.1.568#L710

@tezeb

This comment has been minimized.

Contributor

tezeb commented Mar 10, 2017

Ok, so I managed to (almost) set it up. Unfortunatelly not pullable yet.
Example build result:
https://coveralls.io/builds/10540064

I've also tried codecov(which imho is way more readable), but there is some weird issue with uploading from travis - it forgets about half of files, but the same commands run by hand works as expected.
So example coverage('by hand'):
https://codecov.io/gh/tezeb/QtPass/tree/3054653b800946eeed4bb9c70045e20f0e188da6/src

And the same repo(with a an extra empty commit) from travis:
https://codecov.io/gh/tezeb/QtPass/tree/b42fa3af7d196616f298a7b5c949b982d987dd8a/src

No idea how to fix this, so I guess coveralls will stay.

@tezeb

This comment has been minimized.

Contributor

tezeb commented Mar 11, 2017

Meh... I've found the issue. gcov(in fact gcc) is too old in Ubuntu. Worse yet, it's too old almost everywhere as a commit adding support for generation of gcov files with no input (no gcda, which is created when running) was added about half year ago.
Bleeding-edge problems :D.

Anyway codecov may be available to QtPass when either there is at least a single test-case for each file or ubuntu will rebuild gcc-6, whichever comes first ;).

And the related commit is ie. here:
gcc-mirror/gcc@bd62d8e

@annejan

This comment has been minimized.

Member

annejan commented Mar 12, 2017

Awesome work checking all this stuff out @tezeb

I'll merge your PR as soon as you submit, and we'll switch to codecov when that becomes applicable.

This was referenced Mar 12, 2017

@annejan annejan closed this in #305 Mar 12, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment