Adding support for txt-like JSON report output

[opikalo]

There are currently 3 types of output: txt, xml and html.
However, when I tried to use the output for programmatically processing coverage data downstream, each format offered some kind of challenge. Text output report has special formatting/headers that prevent robust parsing. XML is tailored for Jenkins reporting and does not have the same level of detail as text report, and HTML is mostly used for end-user consumption.

Is there an intermediate format that I can use? Would there be enough interest in JSON report output? Getting the schema versioning right could be a primary challenge.

This is different from the intermediate output in #282

[latk]

Yes! That's something that I would really like to see. A couple of pointers: * If gcovr creates a JSON report, it should be largely compatible with the GCC 9 intermediate format. * See the data model (I think in gcovr/coverage.py) for the data that is currently available * Having JSON output would allow the test suite to be drastically simplified (and sped up) * Being able to read and write JSON would allow for interesting use cases like lcov-style merging of multiple runs. * I want to avoid a future where there would be two different JSON *input* formats (GCC vs gcovr) So this is in many ways a cornerstone feature. However, I'll be quite firm on GCC 9 compatibility. I just saw your #302 but don't know whether that JSON format would be compatible. If we create JSON output, it should have maximum fidelity, not just report derived metrics like coverage percentages.

[opikalo]

When I wrote #302 I was primarily focused on representing the current text report output in a JSON format. It's much smaller task compared to the grand vision for intermediate JSON format and changing all reports to be viewing layer based on it: as you mentioned, it would require a rewrite of the test rig but will simplify things a lot. It looks like #282 is exactly that.

In a way, I think of them as separate features. Could we maybe take baby steps and offer an incremental level of details for the export report? This way multiple developers would be able to pile on, and at some point create all the reference files for the test rig.

Do you know if there is a validator for JSON format compatible with GCC 9.1? I can try to make it happen, but so far I only find some sporadic examples of documentation through implementation.

[latk]

#282 is about reading and using the GCC 9 json intermediate format. For example, this might speed up gcovr significantly. It is not about generating JSON reports, but as outlined above they are linked.

is [there] a validator for JSON format compatible with GCC 9.1?

Unfortunately not. There are the gcov docs, and there is the source code (see file gcc/gcov.c in https://github.com/gcc-mirror/gcc). The JSON format is somewhat in flux.

I'm OK if we only generate a subset of the data structure, or if we introduce our own fields but clearly mark them as extension points. If in doubt, check with @marxin.

Could we maybe take baby steps and offer an incremental level of details for the export report?

“No is temporary, yes is forever.” I'm hesitant to include experiments if there's a good chance they won't work out, because I'll be the one having to keep backwards compatibility with that format.
I would like to avoid a future where there would be competing JSON output formats (unless they have clearly different names, e.g. --json-summary and not just --json).
I'd also be OK if we do get it right from the start and keep GCC JSON compatibility, but start with a very small subset (see above item).

Another alternative would be to create a plugin interface where I can commit to long-term stability, then extra reporters could be implemented externally.

I do not expect you to overhaul the test suite, that is just a long-term vision.

[marxin]

I'm OK if we only generate a subset of the data structure, or if we introduce our own fields but clearly mark them as extension points. If in doubt, check with @marxin.

As mentioned the format is rather new and may be changes in the future. If someone needs an intermediate format, why not using gcov (GCC) directly?