css/WOFF2: Consider providing fonts in FontToInstall/ via @font-face rules

[fred-wang]

The tests in http://www.w3c-test.org/css/WOFF2/ fail if one does not install the support fonts in https://github.com/w3c/web-platform-tests/tree/master/css/WOFF2/support/FontsToInstall ; Would it make sense to serve them via @font-face rules like what is done for acid/acid3/test.html ? Maybe as woff1 fonts for better interoperability ? (Maybe move them to https://github.com/w3c/web-platform-tests/tree/master/fonts/ ?)

cc @khaledhosny @youennf

[khaledhosny]

I think they are used as local fonts to rule-out any web font loading issues.

[fred-wang]

cc'ing @jfkthame for advice on how it is done in mozilla-central.

I think one can assume that at least basic WOFF1 support is available and working before implementing WOFF2. So if WOFF1 fonts fail to load, it seems likely that the same issue will happen for the WOFF2 fonts used in the tests, right?

Would there be any concerns to add a WOFF1 fallback in addition to the local fonts (e.g. slowing down test executions)? At worst, I guess we can install test fonts in WebKit, but it's just extra work and not done the same way for all WebKit ports. Is it how it is done in mozilla-central too?

[litherum]

What web font loading issues are you referring to? The domcontentloaded event isn't fired until web fonts have been downloaded.

[jfkthame]

What web font loading issues are you referring to? The domcontentloaded event isn't fired until web fonts have been downloaded.

I don't think that's necessarily true, is it? Webfont loading may not even be triggered until reflow happens (at which point styles are resolved and the font-matching algorithm determines what fonts are needed for the actual character content of the elements).

[jfkthame]

cc'ing @jfkthame for advice on how it is done in mozilla-central.

In mozilla-central, tests that depend on specific fonts generally load them using @font-face from local files that are checked in to the tree alongside the testcases. In some cases, I think this requires some added machinery to ensure we wait until everything has fully loaded -- especially for tests that are dealing with dynamic changes to styles, etc. (Look for uses of class="reftest-wait" on the root element of mozilla reftests.)

For testing things like WOFF2 support, we'd typically start with a basic test that the format is supported, by checking that a testcase using a WOFF2 font renders differently from a reference that doesn't use it; if that were to fail (because the webfont doesn't load, and we fall back to the default font) then any further tests involving the format would be understood to be invalid, even if they happen to "pass" trivially because fallback renderings match.

[fred-wang]

I've created a branch with my proposal in https://github.com/w3c/web-platform-tests/tree/woff2-fonts-to-install-as-web-fonts. It looks like available-001.xht is not strictly speaking a reftest and should be adjusted a bit if we want to automate the execution.

In mozilla-central, tests that depend on specific fonts generally load them using @font-face from local files that are checked in to the tree alongside the testcases. In some cases, I think this requires some added machinery to ensure we wait until everything has fully loaded -- especially for tests that are dealing with dynamic changes to styles, etc. (Look for uses of class="reftest-wait" on the root element of mozilla reftests.)

So do you mean it make senses to add @font-face rules directly to the test files, as I do in the woff2-fonts-to-install-as-web-fonts?

reftest-wait is also supported in WPT ( http://web-platform-tests.org/writing-tests/reftests.html ) but again if it's needed we should probably do that for the WOFF2 fonts too.

For testing things like WOFF2 support, we'd typically start with a basic test that the format is supported, by checking that a testcase using a WOFF2 font renders differently from a reference that doesn't use it; if that were to fail (because the webfont doesn't load, and we fall back to the default font) then any further tests involving the format would be understood to be invalid, even if they happen to "pass" trivially because fallback renderings match.

Such a test should probably be in WPT too.

[gsnedders]

We should use @font-face for all of these unless we're actually testing something specific to font loading from the system; anything else leads to the tests not being run as they have a manual prerequisite.

I don't think that's necessarily true, is it? Webfont loading may not even be triggered until reflow happens (at which point styles are resolved and the font-matching algorithm determines what fonts are needed for the actual character content of the elements).

AFAICT, every browser has the behaviour such that the load event only fires after all used web fonts have loaded, and reftests don't have their screenshot taken until after the first paint after the load event fires. (Of course, if something happens while the load is being fired to cause a new web font to be loaded, then it won't be.) I think for all the WOFF2 tests we don't have any complex timing here, given they're mostly just "does this font render like this other font"?

[litherum]

@gsnedders is exactly correct.

[fred-wang]

It looks like available-001.xht is not strictly speaking a reftest and should be adjusted a bit if we want to automate the execution.

For the record, I opened #8482

[gsnedders]

@fred-wang I suggest you open a PR for your woff2-fonts-to-install-as-web-fonts branch. As one pre-review comment, you should get rid of <meta name=flags content=font> elements from tests that no longer need fonts to be installed.

[fred-wang]

@gsnedders OK, I opened #8494 and I've done a more extreme change by removing all the "install local fonts" stuff. Tests seem to work as expected when I run them on WebKitGTK.

[svgeesus]

I think they are used as local fonts to rule-out any web font loading issues.

They were done that way when we did the WOFF 1.0 test suite, because webfont loading was a lot less interoperable then.

As you know, the WOFF 2.0 tests were derived from the WOFF 1.0 tests (for the record @khaledhosny did most of that conversion work) but meanwhile the state of webfont loading has improved a lot and we no longer need to worry that some browser doesn't load any webfonts at all.

Also, it that does happen (and it will, in the case of some CSS to PDF formatters for example which lag behind browsers) the tests will not display the PASS or FAIL glyphs; they will display the usual glyphs for the characters P and F.

[svgeesus]

See also #8482 (comment)

[fred-wang]

See also #8482 (comment)

OK, it's fine by me if we wait that WOFF2 is moved to Proposed Recommendation.