Update webidl2.js to v8.1.0

[gsnedders]

THIS MUST BE MERGED WITH "CREATE MERGE COMMIT" WITH HISTORY AS-IS AND NOT SQUASHED OR REBASED (failure to do this will make future updates painful).

---

This change is 

[gsnedders]

Note the first commit here (the squash commit) deletes all of WPT and reintroduces webidl2 at the root of the repo, as git-subtree expects (there should essentially be a linear history of squash commits, occasionally merged into master with the files moved). The previous squash commit has everything at resources/webidl2 because it comes from @jgraham squash-and-merging the previous update at #5665, and this confuses git-subtree massively and I just wasted two hours trying to figure out how to do this. :(

[foolip]

We don't have any tools to judge how much this would break, if anything, so I'm going to try applying the changes to Chromium and seeing if any tests fail there.

[ghost]

Build ERRORED

Started: 2017-11-20 19:05:35
Finished: 2017-11-20 19:54:06

Failing Jobs

• firefox:nightly
• MicrosoftEdge:14.14393

View more information about this build on:

• WPT PR Status
• TravisCI

[foolip]

Trying it in https://chromium-review.googlesource.com/c/chromium/src/+/754234

[foolip]

I just wasted two hours trying to figure out how to do this. :(

Do you have instructions for the next time we do this, to not put it in a weird state again?

[foolip]

This will break lots of stuff, see https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/580021/layout-test-results/results.html

[marcoscaceres]

Just wondering, but how come we don’t “npm install” this instead? Or is adding a dependency on node/npm too much?

[foolip]

Just wondering, but how come we don’t “npm install” this instead? Or is adding a dependency on node/npm too much?

Replacing all of our python infrastructure with node would be kinda nice, but just to clone a repo it'd be a bit of a gratuitous dependency I think. We'd still have to upgrade it once in a while, and we could have a 3-line script for it. Then, figuring out what will break will be the bulk of the work.

[marcoscaceres]

Replacing all of our python infrastructure with node would be kinda nice, but just to clone a repo it'd be a bit of a gratuitous dependency I think.

Absolutely. I was only wondering just for js dependencies that are on npm, and not suggesting we replace any existing python infrastructure.

[foolip]

Absolutely. I was only wondering just for js dependencies that are on npm, and not suggesting we replace any existing python infrastructure.

Sure, that was my half-serious suggestion :)

[gsnedders]

@foolip:

Do you have instructions for the next time we do this, to not put it in a weird state again?

Yes. Don't squash-and-merge or rebase-and-merge, always push the history as git-subtree creates it (either manually or with GitHub "create merge commit"). And then it should just be git subtree pull --prefix=resources/webidl2 https://github.com/w3c/webidl2.js.git v8.1.0.

[jgraham]

Replacing all of our python infrastructure with node would be kinda nice

No it wouldn't.

[foolip]

Replacing all of our python infrastructure with node would be kinda nice

No it wouldn't.

To be clear, I don't actually think we should do that. The bit that would have been nice is one fewer language to understand for contributors.

[marcoscaceres]

To be clear, I don't actually think we should do that. The bit that would have been nice is one fewer language to understand for contributors.

Agree. I've learned Python maybe 3 or 4 times in the last 15 years, but use it so rarely that I always end up forgetting it. I imagine it's the same for most people doing web stuff, with JS being a more natural fit and Node being much more capable nowadays.

At least we should make an to not add new stuff in Python if we can avoid it.

[foolip]

I don't know about that, at this point wpt infra is entirely Python, and doing some new things in another language would end up fairly messy I think, unless very isolated. To really switch away from Python would be a very large undertaking, and not a very good use of time at this point I think. Although, I do know Python somewhat well and mostly like it, so I may not be representative.

[gsnedders]

When Python is the scripting language used by three out of four major browser engines, there's a somewhat strong argument against moving away from it.

As for the regressions:

• We have a number of WebIDL fragments that fail to parse (css/cssom-view/interfaces.html, css/cssom/interfaces.html, gamepad/idlharness.html, html/dom/interfaces.html, html/dom/interfaces.worker.html, mediacapture-streams/MediaDevices-IDL-all.html, mediacapture-streams/MediaDevices-IDL-enumerateDevices.html, selection/interfaces.html, webaudio/the-audio-api/the-delaynode-interface/idl-test.html, webaudio/the-audio-api/the-gainnode-interface/idl-test.html, webrtc/interfaces.https.html). @marcoscaceres can you have a quick look at some of these and figure out whether the WebIDL in the spec doesn't conform, whether the test contains the wrong WebIDL, or whether there's a bug of webidl2.js? I suspect you can do that much quicker than me!
• We have loads of failures like FAIL XMLHttpRequest interface: constant UNSENT on interface object assert_equals: property has wrong value expected (string) "0" but got (number) 0. This is the breaking change from BREAKING CHANGE: argument + default types should be string w3c/webidl2.js#95.

[foolip]

What part of the syntax is problematic in the above mentioned issues?

The first one must be because of array syntax in https://github.com/w3c/web-platform-tests/blob/a24dde9c8969dab3dd422b25902198faf4963823/gamepad/idlharness.html#L55

@gsnedders, do you have an exhaustive list?

[foolip]

I'm fixing the gamepad one in #8168

[foolip]

https://github.com/w3c/web-platform-tests/blob/master/content-security-policy/reporting/securitypolicyviolation-idl.html remains to be done.

[tobie]

Opened whatwg/webidl#477 for a proposed solution to this issue.

[gsnedders]

@foolip so only that remains to be fixed?

[gsnedders]

@foolip If that's all of them now resolved, let's try one more CQ run to make sure nothing has regressed again?

[foolip]

Giving it another try in https://chromium-review.googlesource.com/#/c/chromium/src/+/779581

[foolip]

OK, all the previously noticed issues are resolved now, only this thing remains:
https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/591895/layout-test-results/results.html

--- /b/s/w/ioeX1Xzh/layout-test-results/http/tests/w3c/webperf/approved/navigation-timing/html/idlharness-expected.txt
+++ /b/s/w/ioeX1Xzh/layout-test-results/http/tests/w3c/webperf/approved/navigation-timing/html/idlharness-actual.txt
@@ -55,22 +55,22 @@
 PASS PerformanceNavigation interface object name
 PASS PerformanceNavigation interface: existence and properties of interface prototype object
 PASS PerformanceNavigation interface: existence and properties of interface prototype object's "constructor" property
-PASS PerformanceNavigation interface: constant TYPE_NAVIGATE on interface object
-PASS PerformanceNavigation interface: constant TYPE_NAVIGATE on interface prototype object
-PASS PerformanceNavigation interface: constant TYPE_RELOAD on interface object
-PASS PerformanceNavigation interface: constant TYPE_RELOAD on interface prototype object
-PASS PerformanceNavigation interface: constant TYPE_BACK_FORWARD on interface object
-PASS PerformanceNavigation interface: constant TYPE_BACK_FORWARD on interface prototype object
-PASS PerformanceNavigation interface: constant TYPE_RESERVED on interface object
-PASS PerformanceNavigation interface: constant TYPE_RESERVED on interface prototype object
+FAIL PerformanceNavigation interface: constant TYPE_NAVIGATE on interface object assert_equals: property has wrong value expected (string) "0" but got (number) 0
+FAIL PerformanceNavigation interface: constant TYPE_NAVIGATE on interface prototype object assert_equals: property has wrong value expected (string) "0" but got (number) 0
+FAIL PerformanceNavigation interface: constant TYPE_RELOAD on interface object assert_equals: property has wrong value expected (string) "1" but got (number) 1
+FAIL PerformanceNavigation interface: constant TYPE_RELOAD on interface prototype object assert_equals: property has wrong value expected (string) "1" but got (number) 1
+FAIL PerformanceNavigation interface: constant TYPE_BACK_FORWARD on interface object assert_equals: property has wrong value expected (string) "2" but got (number) 2
+FAIL PerformanceNavigation interface: constant TYPE_BACK_FORWARD on interface prototype object assert_equals: property has wrong value expected (string) "2" but got (number) 2
+FAIL PerformanceNavigation interface: constant TYPE_RESERVED on interface object assert_equals: property has wrong value expected (string) "255" but got (number) 255
+FAIL PerformanceNavigation interface: constant TYPE_RESERVED on interface prototype object assert_equals: property has wrong value expected (string) "255" but got (number) 255
 PASS PerformanceNavigation interface: attribute type
 PASS PerformanceNavigation interface: attribute redirectCount
 PASS PerformanceNavigation must be primary interface of window.performance.navigation
 PASS Stringification of window.performance.navigation
-PASS PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_NAVIGATE" with the proper type
-PASS PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_RELOAD" with the proper type
-PASS PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_BACK_FORWARD" with the proper type
-PASS PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_RESERVED" with the proper type
+FAIL PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_NAVIGATE" with the proper type assert_equals: expected (string) "0" but got (number) 0
+FAIL PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_RELOAD" with the proper type assert_equals: expected (string) "1" but got (number) 1
+FAIL PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_BACK_FORWARD" with the proper type assert_equals: expected (string) "2" but got (number) 2
+FAIL PerformanceNavigation interface: window.performance.navigation must inherit property "TYPE_RESERVED" with the proper type assert_equals: expected (string) "255" but got (number) 255
 PASS PerformanceNavigation interface: window.performance.navigation must inherit property "type" with the proper type
 PASS PerformanceNavigation interface: window.performance.navigation must inherit property "redirectCount" with the proper type
 FAIL Performance interface: existence and properties of interface object assert_equals: prototype of self's property "Performance" is not Function.prototype expected function "function () { [native code] }" but got function "function EventTarget() { [native code] }"

@gsnedders, can you tell if the old or the new state is the correct one?

[foolip]

Hmm, after a quick look at https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/http/tests/w3c/webperf/approved/navigation-timing/html/idlharness.html I'm pretty sure this is a regression. These shouldn't be strings, they should be numbers.

[gsnedders]

@foolip That to me looks like something's wrong on the Chromium side; loading http://web-platform.test:8000/navigation-timing/idlharness.html locally WFM. (Also should probably get rid of those copies of the tests in the Chromium tree.) Did you update idlharness.js when doing the import to run that CQ? 881cd41 changes it in the exact way that would fix this, and last I knew idlharness.js (and testharness.js) weren't auto-imported it.

[foolip]

@gsnedders I only updated the copies of things inside LayoutTests/external/wpt, I'll try also syncing the other idlharness.js.

[foolip]

Actually, I did https://chromium-review.googlesource.com/c/chromium/src/+/781684 instead.

[gsnedders]

@foolip so does this look okay to you then?

[foolip]

Sure, it might break our import but I'm on rotation now and can handle it.