set up unit testing for canvas/drawing functions

[lmccart]

probably using this:
https://github.com/learnboost/node-canvas

set up would include:

1. integrating node-canvas into existing unit test framework
2. creating example tests for a few drawing functions
3. updating testing documentation with any relevant information

[radialglo]

@lmccart
If node-canvas is integrated would the plan be to eventually move all tests to node or have a mix of node and phantomjs?

phantomjs currently supports canvas, so to reduce friction, I can write the tests directly in phantomjs for now?

If it's fine, right now I think I'm going integrate the test to phantomjs. I'm thinking about writing the tests like https://github.com/LearnBoost/node-canvas/blob/master/test/canvas.test.js#L467 The tests should be platform agnostic, so it should be relatively easy to move the tests to node should such a decision be made.

[lmccart]

@radialglo makes total sense, let's start with phantomjs.

rather than testing individual pixels, what do you think about using a checksum instead of testing individual pixels? seems like it would be easier for people to add tests if they just need to make the screenshot, and we could have a generic function that checks they're similar.

i think using crc32 would be the fastest way of comparing, however we might want something that could build in a small threshold to account for renderer differences.
http://stackoverflow.com/questions/18638900/javascript-crc32

in that case, optimal might be just to add up all pixels in each image and compare and see if they come within some threshold difference?

[hamoid]

I believe a checksum might be too strict, as there may be antialiasing
differences in different browsers or OSes.

Would it be possible to subtract the expected from the resulting image?
If they are equal, the result should be pretty close to pure black. One
can then set a threshold when evaluating the blackness of this
difference image.

[lmccart]

@hamoid yes, I agree. I think the crc32 wouldn't work for this. with checksum I was suggesting adding up the pixels and then checking they're within some percent match, rather than 100% same. seems about the same as what you've suggested with subtracting, I think we just need to figure out whether there's any significant performance difference.

[hamoid]

@lmccart If we expect one white pixel on the top left corner, but have it on the bottom right corner instead, wouldn't the sum match, even the image is not the expected one?

[lmccart]

@hamoid good point. I was mostly thinking about a fast "good enough" check, but since a lot of the drawing functions have to do with proper positioning, what I've suggested probably wouldn't be strict enough. I think the subtraction method you suggested makes most sense.

[evhan55]

@hamoid 's difference idea sounds like a good compromise to me, too!

[lmccart]

so I took a first pass at setting this up. the testRender function takes an image file and compares to the current canvas:
https://github.com/lmccart/p5.js/blob/62e8a966f68f2ab6fe31019abe7a939982010a67/src/output/image.js#L11

I think this function could be optimized. and the function probably belongs in a different file.

I used the save() command to save out the image of a test canvas (on chrome, but I think safari is probably a closer render match to phantomjs that mocha is using). and then I'm comparing that image against the canvas in this test here:
https://github.com/lmccart/p5.js/blob/62e8a966f68f2ab6fe31019abe7a939982010a67/test/unit/shape/2d_primitives.js#L16

you can see there's a threshold to account for different renderers, based on checking how much difference there was between mocha and manual/chrome for a rather complex image.
https://github.com/lmccart/p5.js/blob/62e8a966f68f2ab6fe31019abe7a939982010a67/test/unit/shape/renders/random.png

to understand how the testRender function works you can also try it directly in a sketch after creating a test image. something like:

function setup() {
  background(155);
  fill(0);
  ellipse(0, 0, 100, 100);
  // save(); -- first use save to create the png, then swap this line out for testRender() like below
  testRender('ellipse.png', function(match) {
    console.log(match);
  });
}

[evhan55]

I moved it out to its own file, so you can't test it within a sketch anymore:
lmccart@0ce4eff

I had to change the threshold to be more restrictive because it was letting a false line test pass. I changed it to a threshold that works for the current ellipse and line tests but it probably won't be universal either.

Feel free to revert my changes if you prefer it to stay inside of the p5 prototype.

Another way to save out files is to run a test sketch, call window.location = canvas.toDataURL() and save out that image to your disk.

[lmccart]

great, thanks!