Custom reports from streamed test result objects

[abitrolly]

From #968 (comment) users should have an ability to see test results in their preferred format.

For that, there could be templates that receive stream of test result JSON and format the output accordingly.

There should be an ability to create reports in multiple formats without rerunning the test suite. So the stream of test result objects or events could be saved as JSONL for later processing.

[happz]

From #968 (comment) users should have an ability to see test results in their preferred format.

For that, there could be templates that receive stream of test result JSON and format the output accordingly.

There should be an ability to create reports in multiple formats without rerunning the test suite. So the stream of test result objects or events could be saved as JSONL for later processing.

A bit of clarification WRT how tmt does things: we are talking about two places where "result reporting" happens. One is a progress report, emitted by execute plugin while it's running as part of tmt run, this one is now a stream of lines; the other is report step, executed when all testing is done. The former is short-lived, run-time only, and cannot be replayed without re-running tests - run a test, collect its results, report the simplified outcome, move on to next test. The latter can be re-run many times because it depends on already collected results.

First thoughts:

• we can extend the execute step and plugins to use templates when rendering the progress reporting. Today the format is hardcoded, switching it to a template - and letting the user change the template - should provide us a way to introduce different formats of progress reporting into execute. We can have a default template, we can provide TAP template or a template converting tmt.result.Result instances into JSON/YAML, and so on.
• we can add a report plugin that would read collected results and use a given template to render their reporting. Similar to what html already does, but with the ability to emit to stdout.

And both places can use the very same template. The default/TAP/JSON/YAML/whatever else would be them emitted by execute to stderr while tmt run is in action and by tmt run --last report -h new-plugin any time later, for the same price. I like it.