Run reftests on Windows using ANGLE

[jrmuizel]

This should let us catch any ANGLE related problem earlier

[SimonSapin]

We already have a dev-dependency on ANGLE (so building should already be solved), but so far it’s only used in an ANGLE-specific shader compilation test.

[glennw]

This is a duplicate of #1432.

[glennw]

Well, actually the other bug doesn't mention running reftests - I'll re-open this one and close the other.

[SimonSapin]

#2006 (comment)

Wrench uses glutin for initialization, and that one checks for libEGL as of servo/glutin#112 . All you need is to drop Angle's libEGL.dll in the folder with wrench executable.

[SimonSapin]

We already have a dev-dependency on ANGLE (so building should already be solved)

I was probably a bit too optimistic there. The crate at https://github.com/servo/angle self-describes as "GLSL parsing and validation library based on ANGLE" so possibly it doesn’t compile all of the relevant code. It also produces a static library, whereas Glutin uses (effectively) LoadLibraryA("libEGL.dll").

[SimonSapin]

https://crates.io/crates/mozangle + rust-windowing/glutin#998 should enable running Wrench with ANGLE on Windows in non-headless mode.

However headless mode still uses OSMesa, and our Windows CI doesn’t run reftests at all at the moment. @jrmuizel Did you mean this issue to be about running reftests on CI?

[jrmuizel]

Yes, I'd like to have the reftests running in CI on ANGLE

[SimonSapin]

#2529 adds wrench --angle. 7 reftests fail, even with some fuzzy matching:

        reftests\border\border-suite-2.yaml == reftests\border\border-suite-2.png
        reftests\border\dotted-corner-small-radius.yaml == reftests\border\dotted-corner-small-radius.png
        reftests\boxshadow\inset-subpx.yaml == reftests\boxshadow\inset-subpx.png
        reftests\image\downscale.yaml == reftests\image\downscale.png
        reftests\transforms\image-rotated-clip.yaml == reftests\transforms\image-rotated-clip.png
        reftests\transforms\nested-rotate-x.yaml == reftests\transforms\nested-rotate-x.png
        reftests\transforms\nested-preserve-3d.yaml == reftests\transforms\nested-preserve-3d.png

Full output of cargo run --release -- --angle reftest --fuzzy 10: https://gist.github.com/anonymous/fc947365525cd6066b3eb77d10084cdd

[glennw]

I opened up the reftest log, and none of the failing tests look completely wrong. Most of them are a lot of subpixel differences compared to the PNG reference images on Linux, which is somewhat expected.

If we disable those tests on Windows temporarily, by adding platform(max,linux) to the reftest.list file for those tests, we should be able to enable windows reftests in CI.

Does that sound correct to you @SimonSapin ?

[glennw]

Long term, we need to either use PNG images less often, or support generating them per-platform (probably a combination of both).

[glennw]

This happened. We still need to enable some of the PNG reference tests on Windows, but we have ~80% coverage.