Separate HTML runner and HTML reporter?

[platinumazure]

(Ditching the issue template since this is a meta-issue)

Now that we're working on a QUnit Node CLI and integrating with js-reporters, I think we need to see if we can refactor the HTML Reporter space to split that into HTML "runner" and HTML "reporter".

Conceptually, here is what I would like to see long-term:

• HTML runner focuses on running QUnit in a browser, decoupling itself from reporting result. It takes on the outermost container of current HTML reporter but the internals should be black box. HTML runner also handles configuration (including url-based config) and test/module filtering.
• HTML reporter simply outputs the inner divs it currently does for every test.
• Create a new reporter which simply takes plaintext from another reporter and puts it in an HTML block. Voila, HTML runner now integrates with any plaintext reporter.
• Create a strategy for allowing plaintext reporters to gain an HTML/"enhanced" version for better integration? Not sure of the best way to handle that to be honest.

[gibson042]

I like this, and note also the (loosely) related #947.

[leobalter]

Me too. If we split both we might be able adapt into a HTML js-reporter as well.

[Krinkle]

I'm looking into this as part of #1486, and running into an obstacle with testId. We currrently get this from Core and use it in the HTML reporter as part of the HTML element IDs, which are intended to be part of the public API (ref #1751) so that QUnit plugins can use them to display additional visualisations.

For QUnit's own use, we could trivially generate a different set of IDs (e.g. lazily, from the testStart event).

For plugins, we may need to change it so that the plugin reacts to the HTML rather than influencing it from the inside. For example, the way a plugin finds the HTML element currently is via (contextual) assert.test.testId.

Straw-man proposal:

• HTML Reporter: Generate new test IDs here.
• HTML Reporter: Emit custom events (not part of js-reporters/CRI) for the purpose of augment the display of a completed test and a completed assertion. Analogous to QUnit.testDone (CRI testEnd) and QUnit.log (has no CRI event), from which we'd provide references to the DOM elements in question.

Challenges this brings about:

• Can we actually deprecate the core testId property? It seems we'd need to keep this at least privately for the URL parameter and re-running tests. However, "rerun" is an intimate part of the user interface. It's not like the checkboxes and module dropdown that are neatly separated in the would-be "HTML Runner" component.
• How would our HTML Reporter operate based on CRI events whilst still being able to do QUnit-specific things like re-running tests. Maybe it's fine to have inline conditionals that would be QUnit-specific and if it were to be its own project at some point, support for other custom features of individual testing frameworks could be added. Maybe that's not so bad, and still a good enough improvement over the status quo.

Perhaps it's not worth separating the two, but we could still add support for disabling the bulk of the output and running a plain text CRI reporter in its place. In any event, that could of course be a good first step either way.