Support for Karma

[olegskl]

Are there plans integrating Adana with Karma runner?

[izaakschroeder]

@olegskl yes! I would like to support karma due to its massive popularity as a testing framework. I'm not sure how hard it will be to do, and my time has been fairly limited as of late, but it is definitely on the radar. 😄

[olegskl]

I have created a proof-of-concept repository karma-adana-reporter and tested it with my adana-format-text formatter. Here's the result in console (optional) when running it against our codebase with my forked version of babel-plugin-transform-adana:

And it also generates (optionally) a file project/reports/coverage/PhantomJS 1.9.8 (Mac OS X 0.0.0)/text/text.txt with the following contents:

Failed overall test coverage on PhantomJS 1.9.8 (Mac OS X 0.0.0)
------------------------------------------------------------
Statements: 99.69% (covered 1275/1279, threshold 99%)
Branches: 92.87% (covered 391/421, threshold 97%)
Lines: 99.65% (covered 2246/2254, threshold 99%)
Functions: 98.98% (covered 678/685, threshold 99%)
------------------------------------------------------------

I can only say that I feel very happy! The babel plugin works like a charm and although it misses a few statements here and there, in some cases it provides results that are actually correct compared to Istanbul.

There's basically no need to modify the babel plugin to make it work with Karma, however it would be great if formatters could test coverage results against thresholds, like above, which is similar to what karma-coverage is doing.

Also there're a few issues I'm not sure how to deal with:

1. Stats are generated multiple times, by every formatter individually. This has its benefits, but also creates performance issues, especially when formatters need to read source files. It may also lead to different results across different formatters (I see it happening in our existing setup with karma-coverage and Istanbul).
2. I'm not sure how to deal with formatters that may need to output multiple files. Right now formatters return a string, which is great, but how to create multi-file report writers? Do we even need them? I think that html reporter might just write a single html file with JSON data, JS and CSS inlined and minified.
3. I would like to move the repositories to adana-coverage organization when they are a bit more mature (at least when unit tests are added, as there are none yet... yes, I'm that bad!) because I believe it would be easier to discover and maintain. Would that be possible?

There are still bits and pieces of code that aren't covered by the babel plugin. I will go through files individually and try to figure out the non-covered parts.

[izaakschroeder]

This is great! Thanks so much for all the work!

I can only say that I feel very happy! The babel plugin works like a charm and although it misses a few statements here and there, in some cases it provides results that are actually correct compared to Istanbul.

Awesome! I'd actually love to hear about both cases – the issues so I can get them fixed and where it works better than Istanbul since it'll probably help encourage others to use the library 😄 Would be a huge asset!

There's basically no need to modify the babel plugin to make it work with Karma, however it would be great if formatters could test coverage results against thresholds

I agree, though I want to be at least a little careful in this regard – I'm trying not to pigeonhole myself too much as I'd like to keep most of the functionality orthogonal – one small thing just does that one small thing but does it well. Same issue applies to the multi-file formatters. I'll think on that this weekend.

Stats are generated multiple times, by every formatter individually. This has its benefits, but also creates performance issues, especially when formatters need to read source files.

Also agreed. Maybe it'll be possible to make adana-analyze memoized (i.e. cache some number of results). Will also try and think about the best approach to this. It's nice to give each formatter the freedom to do whatever it wants with the raw data, though using analyze is probably an extremely common case, so it makes sense to optimize at least a little for it.

I'm not sure how to deal with formatters that may need to output multiple files. Right now formatters return a string, which is great, but how to create multi-file report writers? Do we even need them? I think that html reporter might just write a single html file with JSON data, JS and CSS inlined and minified.

I have thought about this before and come to the same conclusion as you have regarding html – single output stream is a simple convention that makes it easy to do all sorts of things by chaining multiple simple commands together (UNIX pipe style), which I like. It's entirely possible for a formatter to call fs.createWriteStream or whatever itself. This API might change in the future if there's a big need for multiple files, but I'm kinda happy with its simplicity right now.

I would like to move the repositories to adana-coverage organization when they are a bit more mature (at least when unit tests are added, as there are none yet... yes, I'm that bad!) because I believe it would be easier to discover and maintain. Would that be possible?

Super cool! I'd love to have this as part of the organization. Feel free to transfer one or both repos over and I'll give you ownership of the two and you can tinker there. This project is still pretty young so having more community involvement is a great thing.

Thanks again for all your work, super appreciated! 👌

[olegskl]

I have thought about this before and come to the same conclusion as you have regarding html – single output stream is a simple convention that makes it easy to do all sorts of things by chaining multiple simple commands together (UNIX pipe style), which I like.

I have created another proof-of-concept adana-format-html. It has the same behavior as other formatters, i.e. outputs a string which can be piped into a report.html file. So multiple files aren't an issue for the time being.

We can probably close this issue because there's no need to modify the babel-plugin-transform-adana to make it work with Karma.

[kentcdodds]

Maybe leave it open until there's documentation or a reference to a tool for using this with Karma?