Run tests from command line and save results to a folder?

[jfkthame]

The test page/script is awesome, but I've been thinking it might also be nice (and less resource-hungry for the browser) to have the option of running tests in a "batch" mode from the command line, generating a report similar to the current test page (but as a static file), with the test images and difference saved to an associated folder as a collection of PNGs.

This would allow us to have a "make check" command that runs the tests and reports a summary of the stats, and the results could then be reviewed in the browser at leisure. It might also make it easier to explore alternative image-diffing tools as part of the test script -- it would be nice if we could find a tool that can be "tuned" to report fewer failures where the only discrepancy is along the fringes of areas, while still detecting "real" differences. (E.g. maybe ImageMagick's compare tool could be useful?)

[timdream]

Right. Being able to use other image processing tools (e.g. OpenCV edge detection) would be nicer than trapping ourselves in the browser.

It shouldn't be hard to extract screenshots with SlimerJS.