Add output option to JSON reporter

[munierujp]

Describe the solution you'd like
I'd like to use test report for other applications (e.g. Slack)
So, JSON file is useful.

package.json:

{
  "scripts": {
    "test": "mocha ./test/*-test.js --reporter json --reporter-options output=report.json",
    "posttest": "node slack.js"
  }
}

slack.js:

const readJSONFile = require('./readJSONFile')
const createMessage = require('./createMessage')
const postMessage = require('./postMessage')

const report = await readJSONFile('report.json')
const message = createMessage(report)
await postMessage(message)

Describe alternatives you've considered

• https://github.com/i3oges/mocha-json-file-reporter

[juergba]

@munierujp In your shell you can pipe the report output to a file. You don't need a reporter-option for this. In Windows it is the ">" sign.

[munierujp]

The standard output may occasionally contain noise.
And, xunit reporter supports output option already.

By default, it will output to the console. To write directly to a file, use --reporter-options output=filename.xml.

https://mochajs.org/#xunit

Is there a difference between these reporters?

[jmiller-airfox]

any update here? if you need another use case to add momentum to this ticket: i'm trying to add mocha-multi-reporters to a project so that our custom stdout reporter still shows up on our bitbucket pipeline, but also have a json output blob so we can do more programmatically interesting things with the results outside of the execution of the tests themselves.

e: @munierujp json and xunit reporters are actually implemented rather differently. i'm currently taking a stab at introducing rudimentary output file specification similar to how it's done in the xunit reporter.

e2: @munierujp's solution below does what i need. thanks!

[munierujp]

I've created a third-party reporter for outputting test result as JSON file.

• @mochajs/json-file-reporter

[dorny]

Having both human readable console output and json or xunit file output at same time would be very useful.
It's quite ridiculous you need one 3rd party package to support multiple reporters and another 3rd party package to write JSON report to file. Redirecting whole output to file wouldn't work in this case.

I run into issue because json-file-reporter doesn't include file property like built in json reporter. This is exactly kind of issue that would not happen if built in json reporter had output option.

[juergba]

@dorny yes, you should definitively terminate this ridiculous situation.
What about a PR from yourside? I will be back, to check on your reply.

[dorny]

@juergba sure I will do my best :)

And don't get me wrong. My intension was not to be offensive. I was simply pointing out current situation is not ideal for the user and your suggestion to pipe the output does not cover all use cases.

By the way, I actually doesn't need this for myself.
I'm working on GitHub action test-reporter.
I've originally implemented support for JEST which I needed. But thought it could be useful for others to support mocha too as it's the second most used testing framework.

Meanwhile I've also discovered another option: mochawesome reporter.
It produces JSON and HTML output while you can still specify another reporter for console output. Unfortunately it uses it's own format of JSON output so I would still prefer solution using mocha-multi-reporters and mocha built-in JSON reporter.

[dorny]

@juergba
I took a first look at the code. Changes to json.js reporter are trivial. Basically it would follow the same pattern as is already implemented in xunit reporter.

However I'm not so sure about unit test.
The json.spec.js looks it would benefit from some refactoring. Existing tests are not actually testing the stdout of reporter, instead they look at testResults property set on runner. This property is set here.
Is runner.testResults actually part of API or it's some old hack just for testing purposes? It doesn't look it's used anywhere else.

All other reporter tests are using different pattern - example here.

My idea would be to remove testResults property and rework spec.json.js so stdout is captured, parsed as JSON and then checked if expectations are met.
Another test then would check if the stdout is empty when output option is provided and if correct content would be created on file system (mocked of course).

So should I remove setting of the testResults property in json reporter and refactor the tests as I proposed?
As I'm completely new to mocha codebase I would appreciate guidance. Thanks.

[juergba]

@dorny thank you for taking over.

[...] Is runner.testResults actually part of API [...]?

I blamed lib/reporters/json.js and found #1294, which states:
It also saves the json result back into the runner object to allow a programmatically mocha run to query the output.

So I propose to leave this property as is, but refactor the tests as per your suggestion. Following the xunit implementation seems a good approach to me.

I don't know this part of our codebase well either, and mocks aren't really my friends. So you shouldn't trust my comments completely.

[opub]

It looks like a fix was merged 5 months ago. Is this available now?

[juergba]

@opub no fix has been merged, this PR has never been finished.