Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pointerevents] Stabilize test #19708

Merged

Conversation

@jugglinmike
Copy link
Contributor

jugglinmike commented Oct 15, 2019

Previously, this test conditionally declared a subtest, making it
difficult to compare results between browsers. Depending on the
implementation status, the results could be interpreted in a number of
ways:

  • a failing single-page test
  • a timed out single-page test
  • a completed test with a single failing subtest
  • a completed test with a single passing subtest

Refactor to explicitly opt-in to the single-page test feature and to
consistently report unexpected behavior via unhandled Promise rejections
and uncaught exceptions.

Previously, this test conditionally declared a subtest, making it
difficult to compare results between browsers. Depending on the
implementation status, the results could be interpreted in a number of
ways:

- a failing single-page test
- a timed out single-page test
- a completed test with a single failing subtest
- a completed test with a single passing subtest

Refactor to explicitly opt-in to the single-page test feature and to
consistently report unexpected behavior via unhandled Promise rejections
and uncaught exceptions.
@jugglinmike

This comment has been minimized.

Copy link
Contributor Author

jugglinmike commented Oct 15, 2019

Firefox's performance for this test is unstable, and this patch does not change that (so the failing stability check is expected). This patch does normalize the format of the results reported for Firefox.

@NavidZ

This comment has been minimized.

Copy link
Contributor

NavidZ commented Oct 16, 2019

Thanks. The change looks good if you can get the FF stability bot to agree with it :)

@jugglinmike

This comment has been minimized.

Copy link
Contributor Author

jugglinmike commented Oct 16, 2019

Thanks @NavidZ. I'm not sure we can address Firefox's instability in WPT because it may be due to the implementation. This change is an improvement in any case because it normalizes how results are expressed. With your approval, I can merge it despite the failing stability check.

@NavidZ
NavidZ approved these changes Oct 16, 2019
Copy link
Contributor

NavidZ left a comment

Sure. Let's merge this as you suggested then.

@NavidZ

This comment has been minimized.

Copy link
Contributor

NavidZ commented Oct 16, 2019

I approved it but I'm not sure whether you can still merge it without getting the bot pass. @Hexcles in case anything here can be done. Maybe we need to skip the test for FF or add the failing expectation to make the bot pass then.

@Hexcles

This comment has been minimized.

Copy link
Member

Hexcles commented Oct 16, 2019

This needs to be admin-merged.

@Hexcles Hexcles merged commit 3b1798b into web-platform-tests:master Oct 16, 2019
12 of 13 checks passed
12 of 13 checks passed
Taskcluster (pull_request) TaskGroup: failure
Details
Azure Pipelines Build #20191015.91 succeeded
Details
Azure Pipelines (./wpt test-jobs) ./wpt test-jobs succeeded
Details
Azure Pipelines (affected tests without changes: Safari Technology Preview) affected tests without changes: Safari Technology Preview succeeded
Details
Azure Pipelines (affected tests: Safari Technology Preview) affected tests: Safari Technology Preview succeeded
Details
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests) wpt.fyi hook: safari-preview-affected-tests succeeded
Details
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests-without-changes) wpt.fyi hook: safari-preview-affected-tests-without-changes succeeded
Details
staging.wpt.fyi - chrome[experimental] Chrome results
Details
staging.wpt.fyi - firefox[experimental] Firefox results
Details
staging.wpt.fyi - safari[experimental] Safari results
Details
wpt.fyi - chrome[experimental] Chrome results
Details
wpt.fyi - firefox[experimental] Firefox results
Details
wpt.fyi - safari[experimental] Safari results
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.