Abstract reftests

[kmcallister]

Instead of dumping an image, write out every DOM node's client rects. We can also implement this as a Firefox extension; then we can test Gecko vs. Servo layout without some of the pitfalls mentioned in #1768. Applications:

• Automatically reduce a page's HTML / CSS (see servo/html5ever#5) to determine why the layout is wrong in Servo.
• Track metrics on layout correctness for the top N websites (live or cached).

[kmcallister]

We can do this inside any engine by injecting some JavaScript:

window.addEventListener('load', function () {
    var elements = [].map.call(document.getElementsByTagName("*"), function (elem) {
        var rects = [].map.call(elem.getClientRects(), function (r) {
            return {
                bottom: r.bottom,
                top: r.top,
                left: r.left,
                right: r.right,
            };
        });
        return {
            name: elem.tagName,
            id: elem.id,
            classes: elem.className,
            rects: rects,
        };
    });

    console.log(JSON.stringify({
        viewport: {
            width: window.innerWidth,
            height: window.innerHeight,
        },
        elements: elements,
    }));
});

[kmcallister]

This approach also supports fuzzy matching and intelligent test case reduction.

[kmcallister]

I have a demo of the above in PhantomJS and I'm working with @Manishearth to implement the Servo side with the new "user scripts" option. Then we should be set for automated, headless testing \o/

[SimonSapin]

Would the test’s rectangles be compared to raw rectangles (maybe stored in JSON or something), or to the rectangles of another (reference) HTML document? If the former, these aren’t reftests and shoul probably be called something else. Layout tests maybe?

[kmcallister]

I called them "abstract reftests" because I imagine comparing to a reference document or to the same document in another engine. But we could use the same script for other kinds of testing, as well.

[kmcallister]

I just got this working in Servo. Here's the diff for Acid1.

[nox]

Is this still something we want to do? Couldn't we reuse webrender's own stuff to specify display lists in written form?