Automated Ref Tests

[humphd]

We need automated ref tests (e.g., tests against a reference image) so we can write regression tests for CubicVR.js' drawing code. There is a bunch of prior-art here in the processing.js project:

Test Runner: https://github.com/jbuck/processing-js/blob/develop/test/ref/index.html

Test Builder: https://github.com/jbuck/processing-js/blob/develop/test/ref/ref-test-builder.html

A bunch of work has been done to generalize this by another student: https://github.com/cwdesautels/sundae.js Using this might be a good start.

Basically what we want is the following:

• A simple way to write a small .js file which renders a canvas. Initially this might be static (e.g., only drawing one frame).
• From this page, a way to save a PNG of the canvas (e.g., dataurl). NOTE: we will have to strip the color profile info from these PNGs or they will fail on different platforms with Firefox (talk to jbuck).
• Next we need a manifest that holds our list of tests (tests.js, a json file). It will indicate which test file to run, and which image to compare against.
• Finally, a page that can load tests + images, run the test, and compare the canvas for the live run to the reference image. This is complicated by the fact that perfect pixel comparisons are impossible across browser/platforms. We need to blur the images first, then compare the pixel values (r,g,b,a) with an epsilon value (e.g., a "fudge factor" which says, "as long as it's the same or within 0.07 or something).

The processing.js team (humph, jbuck, pomax) can help you understand how the ref test stuff should work, and ccliffe can answer questions related to CubicVR.js itself.

[humphd]

Here's an example of our processing.js ref test runner in action:

http://scotland.proximity.on.ca/jbuck/processing-js/test/ref/

[ngitalis]

I got this!