Support rules validation in webpack plugin

[chrisands]

Is your feature request related to a problem? Please describe.
It feels unnecessary to additionally install cli only to validate.

Describe the solution you'd like
Ability to pass statoscope.config.js to webpack plugin and get report warns and errors.

Describe alternatives you've considered
Installing cli and run validation only in CI

[smelukov]

If you validate your stats on CI then you install Statoscoope CLI anyway. Don't you?

[chrisands]

Yes, I do. But I can't see errors and warnings in the report, that has been generated with webpack.
And if there were validation then I wouldn't install cli tool.

[smelukov]

@amalitsky this is another issue that is good for contribution :)
I think we can implement that like this:
webpack plugin tries to resolve statoscope.config from pwd (just like cli does)
If there is a config, then try to validate webpack stats with stat-validator and resolved config
Also, we can add config-field to specify a custom path to statoscope.config

[amalitsky]

If I understand correctly, as of now webpack-plugin is meant to be used during the third party application build to create statoscope report next to stats.json file. Is it right?

Now, if the above is correct, let's assume that statoscope validation (not currently there, but is suggested above) performed by webpack-plugin fails for some reason - is plugin expected to fail the third party application build also? Or console output would be sufficient?

Asking because failing the build will be a breaking change and not failing the build could be misleading.

I admit - never worked on webpack plugins before, hence my understanding is limited at this point.

It seems to me that statoscope is capable of three distinct things:

• gathering extra information during the build. Is it even a thing? Or webpack stats file has all we need?
• formatting it into interactive html representation. Including data required for custom requests with jora.
• validation

Is that right?

[rcherny]

I am loving statoscope @smelukov, but for the life of me I can't figure out how to generate the HTML report with the validation messages. I've installed stats-validator-plugin-webpack -- but that usage in the readme with the statoscope.config.js file seems related to the CLI, not the webpack usage. On the sandbox site, there's a sample that includes validation messages, but when I run my webpack build I don't get that, nor does it happen when I use serve or run statoscope validate. I feel like I'm missing something obvious ... 😢

[rcherny]

Ok I got it! I think the sample code has an incorrect value for the reporter option — indicates to use "@statoscope/stats-report" but it should be '@statoscope/stats-validator-reporter-stats-report'.

When I have a moment and am not against a deadline, I'll circle back and try to describe what I did to get it working and what I found misleading in the docs.

Thanks again for the awesome package(s)!

[smelukov]

@rcherny Thanks for your feedback!
There are two steps to get html-report with validation result:

1. Specofy @statoscope/stats-validator-reporter-stats-report as a reporter (e.g. https://github.com/statoscope/statoscope.tech/blob/main/statoscope.config.js#L8)
   Also, you can specify that reporter as @statoscope/stats-report and statoscope will try to use stats-validator-reporter- as a prefix automatically.

2. Run statoscope validate --input=path/to/stats.json and then you’ll get html-report.

If it works not like it described above, please let me know ☺️

[rcherny]

Appreciate the note! Oddly enough, originally when I tried @statoscope/stats-report and also tried to install that, NPM said it didn't exist. I'll try again soon, but I did get it working using stats-validator-reporter-stats-report. Thanks again,

cheers