Fuzzy reftests

[emilio]

We need the ability to run fuzzy reftests to allow testing radial gradients and similar stuff (see #16385).

I don't have a lot of knowledge in the reftest infra, perhaps @jdm does?

This is probably a good project for starters, though I can't mentor it (I don't know where our image comparison lives).

[jdm]

http://web-platform-tests.org/writing-tests/reftests.html says there's no way standardized for fuzzy matching yet. https://lists.w3.org/Archives/Public/www-svg/2016Dec/0018.html is the last time I'm aware this missing feature was brought up. The WPT reference test comparison code just takes a hash of the two screenshots and compares them right now: https://github.com/w3c/wptrunner/blob/ebcd2639205a5268ea41a9389fd1e42746de722d/wptrunner/executors/base.py#L232-L248

[jdm]

cc @gsnedders

[SimonSapin]

http://searchfox.org/mozilla-central/source/layout/tools/reftest/README.txt

2. A test item

   [ <failure-type> | <preference> ]* [<http>] <type> <url> <url_ref>

   where

   a. <failure-type> (optional) is one of the following:

[…]

      fuzzy(maxDiff, diffCount)
          This allows a test to pass if the pixel value differences are <=
          maxDiff and the total number of different pixels is <= diffCount.
          It can also be used with '!=' to ensure that the difference is
          greater than maxDiff.

      fuzzy-if(condition, maxDiff, diffCount)
          If the condition is met, the test is treated as if 'fuzzy' had been
          specified. This is useful if there are differences on particular
          platforms.

[pyfisch]

We need the ability to run fuzzy reftests to allow testing radial gradients and similar stuff

I have written this radial gradient code that seemed to need fuzzy reftests. It turned out that the two images where indeed different and the result of a bug. One included the border in calculations and the other did not. Fuzzy gradients would have masked this issue.

Writing tests with gradients does not require any form of fuzzy matching in my experience.