[Feature] Add extractSummary to test infrastructure

[evenstensberg]

Is your feature request related to a problem? Please describe.
It would be good to have more granular assertions using extractSummary similar to Jest

Describe the solution you'd like

Implement a utility to use extractSummary like this:

https://github.com/jbdemonte/jest/blob/9a90cfee8779ead918d1379a8a9fa33f25030d76/e2e/__tests__/test_name_pattern_skipped.test.js#L19

Describe alternatives you've considered
N/A

Additional context
N/A

[hemal7735]

@evenstensberg what is the expectation here?

[evenstensberg]

More granular test assertions on stdout, look at how jest did it and try to mimic the same behaviour

[hemal7735]

So, the idea is to have only summary, trimming all stacktraces and individual test group results?

Test Suites: 2 failed, 23 passed, 25 total
Tests:       215 passed, 215 total
Snapshots:   94 passed, 94 total
Time:        126.065s, estimated 141s

What is the plan for integrating this with current infra? Are we going to have proxy which can accept params and then run tests, giving results based on params ?

[ematipico]

@hemal7735 Let me give you some context

We recently re rewrote our test suite for the bin cases. Now it's faster, more maintainable and we also have some snapshot tests.
In these tests we basically run webpack in a sub process and we grab it's output and we remove some information that are not needed. Everything is done in this function

https://github.com/webpack/webpack-cli/blob/master/test/testUtils.js#L67-L77

As you can see it is a dumb function and sometimes we don't want to remove of the information that we need (like the hash). This was done poorly but contributors can enhance it.

Instead of stripping out the output before passing it to the test, we'd like to have some functions that we can call inside the single tests like:

• extractSummary
• extractHash
• etc.

That will give us the outupt that we want and make sure that the snapshots pass (still) and enlarge the what we can assert.