JSON Reporter

[dance-cmdr]

Is your feature request related to a problem? Please describe.
We are making a tool that requires stryker report snapshots to express mutant rating over time. Historical mutant rating.

Describe the solution you'd like
Export a stryker report in a json file.

Describe alternatives you've considered
My first take was to use the data from the html-reporter but it was embedding the data in a file and it wasn't nice to take them out.

[dance-cmdr]

I'm working on this one already as part of our innovation day here in DAZN and as one of my personal objectives.

This idea was discussed a while ago here. stryker-mutator/stryker-handbook#25

[dance-cmdr]

Hi there! Just an update and a couple of questions!

I have the draft of the json-reporter here. https://github.com/dance-cmdr/stryker/tree/json-reporter

It is a very simple implementation so far. The only thing it does so far is stringifying mutationTestReportSchema.MutationTestResult and dumping it in a json file.

The solution is modelled after the html-reporter so it will be very familiar to you. I'm running the tests in the package and everything seems fine. However, when I run npm run all, the script fails with the following...

bug.log
lerna ERR! npm run test exited 2 in '@stryker-mutator/karma-runner'
lerna WARN complete Waiting for 3 child processes to exit. CTRL-C to exit immediately.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! stryker-parent@ mocha: `lerna run test --stream --concurrency 4`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the stryker-parent@ mocha script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/alex.spyropoulos/.npm/_logs/2019-11-04T14_06_04_318Z-debug.log
npm ERR! Test failed.  See above for more details.

What I'm doing wrong?

My second question is. I want to tests the json-reporter on a real project. How do you recommend approaching this? I do not understand how to npm link a lerna package.

[nicojs]

Hi @dance-cmdr 👋 thanks for picking this one up ❤

It is a very simple implementation so far. The only thing it does so far is stringifying mutationTestReportSchema.MutationTestResult and dumping it in a json file.

Great, this is exactly what it should do 😅

Test on a real project: you should be able to use npm link (or npm install file:..) . For example (if your project is situated next to stryker on disk):

npm install file:../stryker/packages/core

You could also use a tool like install-local to do an actual production installation of a local package.

As far as "what am I doing wrong", I cannot see it. You're running npm test in the root directory, which will run npm test in the directory of every package. There is something wrong in the karma-runner package, but I cannot see what from your output. If you want, you can simply run npm test in packages/karma-runner to see in detail what is wrong. If you're just adding the JSON reporter to the core package, it makes sense to just run npm test in that package.

[nicojs]

Just took a look in your branch. You've created a separate package for the reporter. While it's not wrong perse, I would like it if it was integrated in the list of reporters coming with stryker out-of-the-box.

This will be easier for us to maintain as well as easier to use.

You can do this by adding the reporter to this directory:

https://github.com/stryker-mutator/stryker/blob/master/packages/core/src/reporters

And declaring it here:

https://github.com/stryker-mutator/stryker/blob/master/packages/core/src/reporters/index.ts

[dance-cmdr]

Thanks @nicojs!

I'll move json-reporter into the out-of-the-box reporters.

That will probably solve most of my problems with the tests. The problem I have is that the tests pass when I run them from within the package, but, the whole collection would fail when I run it fro the root with test:all

[nicojs]

@dance-cmdr are you still working on this issue? Totally fine if you have other priorities now

[dance-cmdr]

Hi @nicojs!
I was busy with some other stuff but I will work on it tomorrow.