Code coverage tool anytime soon?

[sardaukar]

Is it a priority? I assume Crystal needs to provide some sort of hook for a tool like this to work, right?

[asterite]

I believe something like what Go does (source rewrite) can be done in Crystal too.

[bcardiff]

I played a bit in https://github.com/bcardiff/afix making some instrumentation of reachable code and tracing coverage over some statements (ints assignments). There I change the actual source code, but would not be that hard to hook after the parsing and add some instrumentation.

[waj]

There is also this support from LLVM: http://llvm.org/docs/CoverageMappingFormat.html

[akzhan]

Just mention about https://github.com/SlatherOrg/slather as example of publishing llvm-cov results to coverage web sites (like coverall, codecov).

[RX14]

I just looked at what rust uses for coverage, and found kcov, which looks to be a tool which can generate coverage reports from arbitrary precompiled executables just using their debug information. It seems to work for crystal pretty reliably with no modifications (although it confirmed my suspicion that crystal's debug info needs lots more work from the amount of coverage in doc comments). I'll probably end up integrating it into crystal's jenkins build, which I hope will accelerate getting crystal perfectly compatible with kcov.

[watzon]

Any progress on this? Code coverage would be great.

[RX14]

@watzon the kcov solution seems pretty good. The problem is that somehow LLVM garbles crystal's debug info. That bug needs to be worked around and then kcov would be essentially close to perfect.

[watzon]

@RX14 oh awesome. Is there an issue for that?

[RX14]

#5567 and #4718 are relevant.

[anykeyh]

I take opportunity of this thread to advertise since it's exactly what I'm currently working on:

https://github.com/anykeyh/crystal-coverage

Cannot promise it will not fail on your code (still proof of concept) but in the ideal case, it will generate a proper coverage report 👍

[asterite]

Just a note that I tried such approach in the past and compilation times went to the roof, so I literally couldn't reach the end of compilation to test it in a real project other than a hello word (or was it in a hello world?)

At least my approach was to add some method call at the beginning and end6of each method. If that's your case, don't hold your hopes tok high.

The proper way to do this is using LLVM's coverage feature, which I think @RX14 has done in another issue (as a proof of concept).