Code Coverage Visualization

[psygo]

1. Is your feature request related to a problem? Please describe it

I've been working on a package that has hundreds of tests, so an easy way of visualizing code coverage would be incredibly handy.

2. Describe the solution you'd like

I would like to run my tests with, say, a .vscode configuration with an lcov.info output which would automatically be recognized by VS Code and highlighted on the respective editors with either red or green.

3. Describe alternatives you've considered

I've tried many different solutions in the past few days — months actually — but none of them worked as the ideal one described above:

• flutter test --coverage --coverage-path=lcov.info does work to generate the necessary file, but it's clunky to have to visualize it through a 3rd party program such as genhtml, all the more if you're on Windows.
  • And it does need Flutter in the end, which should not be necessary if you're working on pure Dart...
• IntelliJ would supposedly work ideally, but I just can't seem to enable the Run with Coverage button on mine, even after installing the test_coverage package.
  • Though one person on Gitter told me he has it working on his IntelliJ.
• Both the coverage and the test_coverage packages offer something close to what I described above, but their solutions are way clunkier — and on Windows they are tough to set up...
• codecov.io is an alternative with a 3rd party, but it's annoying to have to handle this externally when the editor offers a much more flexible and faster experience.

[HerrNiklasRaab]

This would be so cool :)

[DanTup]

Thanks for the request! Though I think this might make more sense as a separate extension. Rendering code coverage from an lcov file would be useful for other languages too, so building it in here seems a bit restrictive.

I found an existing extension (https://marketplace.visualstudio.com/items?itemName=alexdima.vscode-lcov - created by one of the VS Code devs), though as far as I can tell, it's just rendering HTML reports rather than in the editor. Has anyone tried it out to see how it feels?

[DanTup]

Slightly related: #653.

[psygo]

I've created a StackOverflow question based on this.

I've also added a mention to the problem of ambiguous coverage, which I don't how codecov.io or the lcov.info file deal with. For example, if one folder tests stuff that indirectly calls another folder, does that count as coverage for the indirectly called folder as well? That's almost always undesirable.

[athomas]

Even if this doesn't become a feature of the extension itself, it would be really nice to be told how to do this on dartcode.org.

[athomas]

Here's what worked for me (slightly modified version of the code posted here):

dart pub global activate coverage
dart test --coverage="coverage"
format_coverage --lcov --in=coverage --out=lcov.info --packages=.packages --report-on=lib

Then I used the Coverage Gutters to visualize the result in vscode (gets me some green lines for code covered by my tests).

[timsneath]

To add to @athomas' comment, now you can just run test_with_coverage and it will automatically generate an lcov.info file with less CLI ceremony. I also use the Coverage Gutters extension very effectively, although there are other VSCode coverage extensions available that others might prefer.

We should probably shrink this feature down to a single new VSCode command "Test with Coverage", which would run the above command line.

[DanTup]

We should probably shrink this feature down to a single new VSCode command "Test with Coverage", which would run the above command line.

Is it reasonable (expected?) to just run that command and generate coverage for the whole project without support for debugging, running tests selectively, or having the results show up in the test runner? To support those things, we usually run tests in special ways and it doesn't seem like these commands support that.

Is test_with_coverage specific to the Dart VM or can it be used for Flutter too? I can't find an examples in the docs that specifically mention Flutter, although the package is marked as Flutter compatible in Pub.

[connor4312]

Fyi, VS Code is finalizing native support for test coverage in the upcoming 1.88 release microsoft/vscode#123713