Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for txt-like JSON report output #301

Closed
opikalo opened this issue Apr 2, 2019 · 4 comments

Comments

@opikalo
Copy link

@opikalo opikalo commented Apr 2, 2019

There are currently 3 types of output: txt, xml and html.
However, when I tried to use the output for programmatically processing coverage data downstream, each format offered some kind of challenge. Text output report has special formatting/headers that prevent robust parsing. XML is tailored for Jenkins reporting and does not have the same level of detail as text report, and HTML is mostly used for end-user consumption.

Is there an intermediate format that I can use? Would there be enough interest in JSON report output? Getting the schema versioning right could be a primary challenge.

This is different from the intermediate output in #282

@latk

This comment has been minimized.

Copy link
Member

@latk latk commented Apr 2, 2019

@opikalo

This comment has been minimized.

Copy link
Author

@opikalo opikalo commented Apr 2, 2019

When I wrote #302 I was primarily focused on representing the current text report output in a JSON format. It's much smaller task compared to the grand vision for intermediate JSON format and changing all reports to be viewing layer based on it: as you mentioned, it would require a rewrite of the test rig but will simplify things a lot. It looks like #282 is exactly that.

In a way, I think of them as separate features. Could we maybe take baby steps and offer an incremental level of details for the export report? This way multiple developers would be able to pile on, and at some point create all the reference files for the test rig.

Do you know if there is a validator for JSON format compatible with GCC 9.1? I can try to make it happen, but so far I only find some sporadic examples of documentation through implementation.

@latk

This comment has been minimized.

Copy link
Member

@latk latk commented Apr 3, 2019

#282 is about reading and using the GCC 9 json intermediate format. For example, this might speed up gcovr significantly. It is not about generating JSON reports, but as outlined above they are linked.

is [there] a validator for JSON format compatible with GCC 9.1?

Unfortunately not. There are the gcov docs, and there is the source code (see file gcc/gcov.c in https://github.com/gcc-mirror/gcc). The JSON format is somewhat in flux.

I'm OK if we only generate a subset of the data structure, or if we introduce our own fields but clearly mark them as extension points. If in doubt, check with @marxin.

Could we maybe take baby steps and offer an incremental level of details for the export report?

“No is temporary, yes is forever.” I'm hesitant to include experiments if there's a good chance they won't work out, because I'll be the one having to keep backwards compatibility with that format.
I would like to avoid a future where there would be competing JSON output formats (unless they have clearly different names, e.g. --json-summary and not just --json).
I'd also be OK if we do get it right from the start and keep GCC JSON compatibility, but start with a very small subset (see above item).

Another alternative would be to create a plugin interface where I can commit to long-term stability, then extra reporters could be implemented externally.

I do not expect you to overhaul the test suite, that is just a long-term vision.

@marxin

This comment has been minimized.

Copy link

@marxin marxin commented Apr 3, 2019

I'm OK if we only generate a subset of the data structure, or if we introduce our own fields but clearly mark them as extension points. If in doubt, check with @marxin.

As mentioned the format is rather new and may be changes in the future. If someone needs an intermediate format, why not using gcov (GCC) directly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.