Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[testharness.js] Don't complete after allowed exc. #19612
The following tests intentionally produce an uncaught exception and then define
testharness.js immediately transitions to "complete" and ignores the
Today, the tests referenced in the commit message are reported as passing single-page tests. Once we've completed the implementation of WPT RFC 28, they will instead be reported as test harness errors.
As an alternative to this patch, we could modify each of the tests. If a subtest is defined before the exception is thrown, then all the subtests are reported as expected. (We could, for example, declare the existing synchronous subtests as asynchronous subtests instead.)
Changing the harness is definitely riskier, but it also seems more correct to me.
This change could interfere with single-page tests that use
All three define subtests indirectly through helper functions.
The following tests intentionally produce an uncaught exception and then define subtests: - html/semantics/scripting-1/the-script-element/module/error-and-slow-dependency.html - html/webappapis/scripting/processing-model-2/window-onerror-parse-error.html - html/webappapis/scripting/processing-model-2/window-onerror-runtime-error.html - html/webappapis/scripting/processing-model-2/window-onerror-runtime-error-throw.html testharness.js immediately transitions to "complete" and ignores the subsequent subtests.
This patch caused a failure in the testharness.js test suite, and that failure demonstrated an edge case I hadn't considered. After looking in to it further (research summarized below), it looks like the edge case was limited to the infrastructure test itself. I don't think it demonstrates anything unsound about this patch.
It's possible that worker-based tests (which require
It doesn't look like this is a problem, though. There are six worker sources which enable
With the exception of the testharness.js test mentioned above, all of them invoke
There are three multi-global tests that use
For these, the
So I see a Fx run; the wpt.fyi diff is https://wpt.fyi/results/?diff&filter=ADC&run_id=321040007&run_id=344290004 Are any of those changes worrying?