Finer control over logging and reporting

[MatthewHambley]

At the moment "systest" likes to take control of logging and report generation. For our use it would be better if we could control more finally what is logged and what is reported.
For instance we would like to be able to control logging to file on and off. We would also like to be able to choose whether the graph reports are produced. Our GitHub action environments do not have "dot" so that stage always fails.
Before I undertake any work, is this something which would fit with the model of operation intended for this framework?

[eerimoq]

systest does not configure logging by default. It's the users decision whether to call configure_logging(), or to configure logging directly using the standard library logging module.

We could make dot optional.

[MatthewHambley]

The problem I'm interested in tackling is that if the user does not call configure_logging() they get no terminal output and no log file. If they do call that function then they get both. If they want only one then they have to, as you say, do a load of manual setup. That manual setup involves copying code already existing in systest and which contains quite a lot of knowledge about the internal workings of the library.
What I'm imagining is finer control to support those of us who want terminal output but not a log file. Others may want it the other way round and they are also not well served by the current solution.
As far as reporting is concerned I wonder if the more flexible solution is to have the Sequencer return a result object from its run() method. This could then be passed to report generators. Alternatively, and preserving the existing run() API, the report generators could be converted to objects which are then registered with the sequencer. That way control over what reports are generated is a matter of which objects you register with it.

[eerimoq]

We could add keyword parameters to configure_logging() to control where to log.

Does summary_json() suite your report needs?