Enable upstream bluetooth tests #22420

cdeler · 2018-12-11T18:45:11Z

I enabled bluetooth WPT tests

But I'm still worry that a lot of tests failed (more than 150). I decreased them by

prefs: ["dom.bluetooth.enabled:true"]

But it's still a lot of failed tests

Checks

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #20437 (GitHub issue number if applicable)

There are tests for these changes OR
These changes do not require tests because the changes are tests

This change is

highfive · 2018-12-11T18:45:20Z

Thanks for the pull request, and welcome! The Servo team is excited to review your changes, and you should hear from @avadacatavra (or someone else) soon.

highfive · 2018-12-11T18:45:21Z

Heads up! This PR modifies the following files:

@jgraham: tests/wpt/include.ini

servo-wpt-sync · 2018-12-11T18:46:24Z

Opened new PR for upstreamable changes.

Completed upstream sync of web-platform-test changes at web-platform-tests/wpt#14468.

jdm · 2018-12-14T17:53:38Z

r? @jdm

jdm

Currently the tests are running on the non-webdriver test runner, but if we actually want to support the testdriver.js api one day (since the tests require it), we will need to use the webdriver-enabled test runner (servodriver) instead.

jdm · 2018-12-14T17:57:51Z

tests/wpt/include.ini

@@ -5,6 +5,8 @@ skip: true
  skip: false
 [2dcontext]
  skip: false
+[bluetooth]
+  skip: false


Let's remove this. The test expectations won't match when run in the non-servodriver test runs.

jdm · 2018-12-14T17:57:51Z

tests/wpt/web-platform-tests/tools/wptrunner/wptrunner/executors/executorservo.py

@@ -33,6 +33,7 @@ def write_hosts_file(config):


 class ServoTestharnessExecutor(ProcessTestExecutor):
+    supports_testdriver = True


Let's move this to executorservodriver.py instead.

This has not been addressed.

jdm · 2018-12-14T17:57:51Z

etc/ci/buildbot_steps.yml

@@ -63,6 +63,14 @@ mac-rel-css2:
  - ./mach test-wpt --release --processes 4 --total-chunks 6 --this-chunk 6 --log-raw test-wpt.log --log-errorsummary wpt-errorsummary.log --always-succeed
  - ./mach filter-intermittents wpt-errorsummary.log --log-intermittents intermittents.log --log-filteredsummary filtered-wpt-errorsummary.log --tracker-api default --reporter-api default

+mac-rel-bluetooth:


Instead of a different job, please add this to an existing mac job that runs WPT tests.

Ok, I've moved the execution to mac-rel-wpt4

jdm · 2018-12-14T17:57:51Z

etc/ci/buildbot_steps.yml

+  - ./mach clean-nightlies --keep 3 --force
+  - ./mach clean-cargo-cache --keep 3 --force
+  - env PKG_CONFIG_PATH=/usr/local/opt/zlib/lib/pkgconfig ./mach build --release
+  - ./mach test-wpt --release --processes=8 --log-raw test-bluetooth.log --log-errorsummary bluetooth-errorsummary.log --always-succeed bluetooth


We will need to pass --product=servodriver with the other changes that I recommended.

This still has not been addressed.

Yes, thank you, Im sorry for delay

servo rebuild takes a while after my rebase to upstream; I just wanted to get #22411 in my repo

It's done, thank you

I've also added --headless to the bluetooth tests execution

servo-wpt-sync · 2018-12-14T20:56:05Z

Transplanted upstreamable changes to existing PR.

Completed upstream sync of web-platform-test changes at web-platform-tests/wpt#14468.

servo-wpt-sync · 2018-12-15T16:26:47Z

Transplanted upstreamable changes to existing PR.

Completed upstream sync of web-platform-test changes at web-platform-tests/wpt#14468.

servo-wpt-sync · 2018-12-15T18:22:28Z

Transplanted upstreamable changes to existing PR.

Completed upstream sync of web-platform-test changes at web-platform-tests/wpt#14468.

cdeler · 2018-12-15T19:29:23Z

@jdm I've fixed the requested changes,

But seems like one of CI builds failed.

I'm trying to figure out whats wrong with the broken test


        #20437 enabled bluetooth wpt tests in config


        #20437 added supports_testdriver to ServoTestharnessExecutor


        #20437 enable bluetooth for bt tests


        #20437 added tests results for wpt bluetooth tests


        #20437 added a special bluetooth tests to CI


        #20437 resolved PR issues


        #20437 moved supports_testdriver from ServoTestharnessExecutor to Ser…

…voWebDriverTestharnessExecutor


        #20437 now bluetooth tests are executed with product=servodriver


        #20437 tried to fix CI build

servo-wpt-sync · 2018-12-16T16:13:12Z

Transplanted upstreamable changes to existing PR.

Completed upstream sync of web-platform-test changes at web-platform-tests/wpt#14468.

jdm · 2018-12-17T14:09:43Z

@bors-servo r+

bors-servo · 2018-12-17T14:09:44Z

📌 Commit 1b6a36b has been approved by jdm

bors-servo · 2018-12-17T14:09:53Z

⌛ Testing commit 1b6a36b with merge 7f64746...


        Auto merge of #22420 - cdeler:enable-upstream-bluetooth-tests, r=jdm

Enable upstream bluetooth tests I enabled bluetooth WPT tests But I'm still worry that a lot of tests failed (more than 150). I decreased them by ``` prefs: ["dom.bluetooth.enabled:true"] ``` But it's still a lot of failed tests Checks --- - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [X] These changes fix #20437 (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [X] These changes do not require tests because the changes are tests  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22420)

bors-servo · 2018-12-17T14:39:58Z

💔 Test failed - status-taskcluster

cdeler · 2018-12-17T14:54:01Z

💔 Test failed - status-taskcluster

I'm not sure that the bluetooth tests enabling could affect tests like:

/quirks/unitless-length/limited-quirks.html
/xhr/open-url-multi-window.htm

(newertheless supports_testdriver = True could)

But some of the tests like /custom-elements/parser/parser-sets-attributes-and-children.html are failing in other branches (e.g. cdeler/implement-DOMException-constructor or master)

jdm · 2018-12-17T15:00:37Z

Yeah, our taskcluster test runner is having big problems right now. This failure isn't caused by these changes.

cdeler · 2018-12-19T11:26:29Z

@jdm Hello,
Do you know, what should I do to help merging the PR?

jdm · 2018-12-19T12:36:36Z

There is nothing you can or should do. #22493 is my attempt to improve the situation.

jdm · 2018-12-22T04:06:22Z

@bors-servo retry

bors-servo · 2018-12-22T07:50:25Z

⌛ Testing commit 1b6a36b with merge 12d0a0b...


        Auto merge of #22420 - cdeler:enable-upstream-bluetooth-tests, r=jdm

Enable upstream bluetooth tests I enabled bluetooth WPT tests But I'm still worry that a lot of tests failed (more than 150). I decreased them by ``` prefs: ["dom.bluetooth.enabled:true"] ``` But it's still a lot of failed tests Checks --- - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [X] These changes fix #20437 (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [X] These changes do not require tests because the changes are tests  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22420)

bors-servo · 2018-12-22T09:24:32Z

☀️ Test successful - android-mac, arm32, arm64, linux-rel-css, linux-rel-wpt, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, magicleap, status-taskcluster
Approved by: jdm
Pushing 12d0a0b to master...

servo-wpt-sync · 2018-12-22T09:24:53Z

Error syncing changes upstream. Logs saved in error-snapshot-1545470803628.

highfive assigned avadacatavra Dec 11, 2018

highfive added the S-awaiting-review label Dec 11, 2018

servo-wpt-sync mentioned this pull request Dec 11, 2018

Enable upstream bluetooth tests web-platform-tests/wpt#14468

Closed

cdeler mentioned this pull request Dec 12, 2018

Enable upstream bluetooth tests #20437

Closed

highfive assigned jdm and unassigned avadacatavra Dec 14, 2018

jdm requested changes Dec 14, 2018

View changes

jdm added S-needs-code-changes and removed S-awaiting-review labels Dec 14, 2018

highfive added S-awaiting-review and removed S-needs-code-changes labels Dec 14, 2018

jdm added S-needs-code-changes and removed S-awaiting-review labels Dec 14, 2018

cdeler force-pushed the cdeler:enable-upstream-bluetooth-tests branch from 91e9a03 to cb9b9b5 Dec 15, 2018

highfive added S-awaiting-review and removed S-needs-code-changes labels Dec 15, 2018

cdeler added 6 commits Dec 11, 2018

#20437 enabled bluetooth wpt tests in config

700a38c

#20437 added supports_testdriver to ServoTestharnessExecutor

2544672

#20437 enable bluetooth for bt tests

3fdc514

#20437 added tests results for wpt bluetooth tests

9d84311

#20437 added a special bluetooth tests to CI

e0030ea

#20437 resolved PR issues

84aca5a

cdeler added 3 commits Dec 15, 2018

#20437 moved supports_testdriver from ServoTestharnessExecutor to Ser…

8f82574

…voWebDriverTestharnessExecutor

#20437 now bluetooth tests are executed with product=servodriver

c408904

#20437 tried to fix CI build

Loading status checks…

1b6a36b

cdeler force-pushed the cdeler:enable-upstream-bluetooth-tests branch from 4022ba6 to 1b6a36b Dec 16, 2018

highfive added S-awaiting-merge and removed S-awaiting-review labels Dec 17, 2018

highfive added S-tests-failed and removed S-awaiting-merge labels Dec 17, 2018

highfive added S-awaiting-merge and removed S-tests-failed labels Dec 22, 2018

bors-servo merged commit 1b6a36b into servo:master Dec 22, 2018
2 of 3 checks passed

2 of 3 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

highfive removed the S-awaiting-merge label Dec 22, 2018

jdm mentioned this pull request Dec 22, 2018

Implement test_driver support #22532

Open

servo / servo

Enable upstream bluetooth tests #22420

Enable upstream bluetooth tests #22420

cdeler commented Dec 11, 2018 •

edited by SimonSapin

highfive commented Dec 11, 2018

highfive commented Dec 11, 2018

servo-wpt-sync commented Dec 11, 2018

jdm commented Dec 14, 2018

jdm left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

servo-wpt-sync commented Dec 14, 2018

servo-wpt-sync commented Dec 15, 2018

servo-wpt-sync commented Dec 15, 2018

cdeler commented Dec 15, 2018

servo-wpt-sync commented Dec 16, 2018

jdm commented Dec 17, 2018

bors-servo commented Dec 17, 2018

bors-servo commented Dec 17, 2018

bors-servo commented Dec 17, 2018

cdeler commented Dec 17, 2018

jdm commented Dec 17, 2018

cdeler commented Dec 19, 2018

jdm commented Dec 19, 2018 •

edited

jdm commented Dec 22, 2018

bors-servo commented Dec 22, 2018

bors-servo commented Dec 22, 2018

servo-wpt-sync commented Dec 22, 2018

		@@ -33,6 +33,7 @@ def write_hosts_file(config):


		class ServoTestharnessExecutor(ProcessTestExecutor):
		supports_testdriver = True

servo / servo

Join GitHub today

Enable upstream bluetooth tests #22420

Enable upstream bluetooth tests #22420

Conversation

cdeler commented Dec 11, 2018 • edited by SimonSapin

Checks

highfive commented Dec 11, 2018

highfive commented Dec 11, 2018

servo-wpt-sync commented Dec 11, 2018

jdm commented Dec 14, 2018

jdm left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

cdeler Dec 14, 2018 • edited

This comment has been minimized.

servo-wpt-sync commented Dec 14, 2018

servo-wpt-sync commented Dec 15, 2018

servo-wpt-sync commented Dec 15, 2018

cdeler commented Dec 15, 2018

servo-wpt-sync commented Dec 16, 2018

jdm commented Dec 17, 2018

bors-servo commented Dec 17, 2018

bors-servo commented Dec 17, 2018

bors-servo commented Dec 17, 2018

cdeler commented Dec 17, 2018

jdm commented Dec 17, 2018

cdeler commented Dec 19, 2018

jdm commented Dec 19, 2018 • edited

jdm commented Dec 22, 2018

bors-servo commented Dec 22, 2018

bors-servo commented Dec 22, 2018

servo-wpt-sync commented Dec 22, 2018

cdeler commented Dec 11, 2018 •

edited by SimonSapin

cdeler Dec 14, 2018 •

edited

jdm commented Dec 19, 2018 •

edited