Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[testharness.js] introduce `assert_precondition` #19993

Merged
merged 2 commits into from Nov 8, 2019
Merged

Conversation

@foolip
Copy link
Member

foolip commented Oct 30, 2019

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.

@foolip foolip force-pushed the foolip/assert_precondition branch 3 times, most recently from 4fdb848 to 3e9e171 Oct 30, 2019
@wpt-pr-bot wpt-pr-bot requested review from gsnedders, jgraham and jugglinmike Nov 2, 2019
@foolip foolip marked this pull request as ready for review Nov 4, 2019
@wpt-pr-bot wpt-pr-bot requested a review from zqzhang Nov 4, 2019
@wpt-pr-bot wpt-pr-bot added the docs label Nov 4, 2019
@foolip
Copy link
Member Author

foolip commented Nov 4, 2019

Copy link
Member Author

foolip left a comment

Bringing a detail to the attention of RFC reviewers coming from web-platform-tests/rfcs#16 (comment).

@@ -3671,16 +3692,19 @@ policies and contribution forms [3].
var tests = new Tests();

if (global_scope.addEventListener) {
var error_handler = function(message, stack) {
var error_handler = function(error, message, stack) {

This comment has been minimized.

Copy link
@foolip

foolip Nov 6, 2019

Author Member

There might be reasonable disagreement to be had about this one, whether an unwrapped precondition_failed should be an error or not. And also whether it really makes sense for allow_uncaught_exception to allow failing asserts as is currently the case.

web-platform-tests/rfcs#16 (comment)

@@ -792,8 +792,10 @@ policies and contribution forms [3].

function done() {
if (tests.tests.length === 0) {
tests.status.status = tests.status.ERROR;
tests.status.message = "done() was called without first defining any tests";
if (tests.status.status === null) {

This comment has been minimized.

Copy link
@foolip

foolip Nov 6, 2019

Author Member

This extra check is a workaround for #20036 (comment) but it might change, I'll use whatever change @jugglinmike lands to fix it.

resources/testharness.js Outdated Show resolved Hide resolved
@jgraham
jgraham approved these changes Nov 6, 2019
@foolip foolip force-pushed the foolip/assert_precondition branch from c223650 to c03ce9b Nov 6, 2019
@foolip
Copy link
Member Author

foolip commented Nov 8, 2019

With #20168 now merged I'll rebase this.

foolip added 2 commits Oct 25, 2019
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
This demonstrates the use of `assert_precondition` for subtests.
@foolip foolip force-pushed the foolip/assert_precondition branch from d9f0280 to f378614 Nov 8, 2019
@foolip
Copy link
Member Author

foolip commented Nov 8, 2019

Alright, given no concerns voices about the behavior of a bare assert_precondition outside of setup or a test, I'll go ahead and merge this. As we change tests to use this we might discover that this wasn't quite right and can change it.

@foolip foolip merged commit a90e89f into master Nov 8, 2019
21 checks passed
21 checks passed
build-and-publish
Details
build-and-tag
Details
Azure Pipelines Build #20191108.20 succeeded
Details
Azure Pipelines (./wpt test-jobs) ./wpt test-jobs succeeded
Details
Azure Pipelines (affected tests without changes: Safari Technology Preview) affected tests without changes: Safari Technology Preview succeeded
Details
Azure Pipelines (affected tests: Safari Technology Preview) affected tests: Safari Technology Preview succeeded
Details
Azure Pipelines (infrastructure/ tests: macOS) infrastructure/ tests: macOS succeeded
Details
Azure Pipelines (tools/ unittests: Windows Python 3) tools/ unittests: Windows Python 3 succeeded
Details
Azure Pipelines (tools/ unittests: Windows) tools/ unittests: Windows succeeded
Details
Azure Pipelines (tools/ unittests: macOS) tools/ unittests: macOS succeeded
Details
Azure Pipelines (tools/wpt/ tests: Windows) tools/wpt/ tests: Windows succeeded
Details
Azure Pipelines (tools/wpt/ tests: macOS) tools/wpt/ tests: macOS succeeded
Details
Azure Pipelines (tools/wptrunner/ unittests: Windows) tools/wptrunner/ unittests: Windows succeeded
Details
Azure Pipelines (tools/wptrunner/ unittests: macOS) tools/wptrunner/ unittests: macOS succeeded
Details
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests) wpt.fyi hook: safari-preview-affected-tests succeeded
Details
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests-without-changes) wpt.fyi hook: safari-preview-affected-tests-without-changes succeeded
Details
Community-TC (pull_request) TaskGroup: success
Details
Taskcluster (pull_request) TaskGroup: success
Details
wpt.fyi - chrome[experimental] Chrome results
Details
wpt.fyi - firefox[experimental] Firefox results
Details
wpt.fyi - safari[experimental] Safari results
Details
@foolip foolip deleted the foolip/assert_precondition branch Nov 8, 2019
bors-servo added a commit to servo/servo that referenced this pull request Nov 15, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 15, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 15, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 15, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 16, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 16, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
bors-servo added a commit to servo/servo that referenced this pull request Nov 16, 2019
Address upstream WPT test harness changes

web-platform-tests/wpt#19449 and web-platform-tests/wpt#19993 caused some of our proprietary tests to silently start to fail. This addresses all of the breakage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

5 participants
You can’t perform that action at this time.