[idlharness] Test class string of prototype objects

[ExE-Boss]

This has been specified in whatwg/webidl#357.

---

See also: #23206

[foolip]

Will this introduce new failures to lots of tests that previously had no failing subtests?

[Ms2ger]

Looks like this breaks some unit tests:

https://community-tc.services.mozilla.com/tasks/V97XKsF0R5uQgpgUykVQ6A/runs/0/logs/https%3A%2F%2Fcommunity-tc.services.mozilla.com%2Fapi%2Fqueue%2Fv1%2Ftask%2FV97XKsF0R5uQgpgUykVQ6A%2Fruns%2F0%2Fartifacts%2Fpublic%252Flogs%252Flive.log

E     -                        {u'message': u'assert_class_string: class string of Window.prototype expected "[object Window]" but got "[object WindowProperties]"',
E     +                        {u'message': None,
E     u'name': u'Window interface: existence and properties of interface prototype object',
E     u'properties': {},
E     -                         u'status_string': u'FAIL'},
E     ?                                             ^ ^^
E     +                         u'status_string': u'PASS'},
E     ?                                             ^ ^^

[Ms2ger]

Will this introduce new failures to lots of tests that previously had no failing subtests?

Good question. Possibly it will cause all idlharness tests to fail in Firefox; I don't know how many are fully green at this point.

[ExE-Boss]

@Ms2ger
Actually, Firefox has had this fixed as of version 78 in bug 1277799.

Also, that got "[object WindowProperties]" for Window.prototype error appears to be an implementation bug, since the class string is specified for the prototype, regardless of whether it’s a global object.

[Ms2ger]

The unit tests need fixing. Looks like CI runs them as

+ ./wpt install firefox browser --destination /home/test
+ ./wpt install firefox webdriver --destination /home/test/firefox
+ export PATH=/home/test/firefox:/home/test/bin:/home/test/bin:/home/test/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/home/test/firefox:/home/test/bin:/home/test/bin:/home/test/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ cd /home/test/web-platform-tests/tools/ci/../../resources/test
+ tox -- --binary=/home/test/browsers/nightly/firefox/firefox

[ExE-Boss]

I have filed https://bugzilla.mozilla.org/show_bug.cgi?id=1688198 to get this fixed in Firefox.

[foolip]

@ExE-Boss do you intend to block this on https://bugzilla.mozilla.org/show_bug.cgi?id=1688198?

Digging into the Chrome regression in https://github.com/web-platform-tests/wpt/pull/27239/checks?check_run_id=1784377402 would be good, but it looks like no changes in Safari test results, which is good.

[ExE-Boss]

Looks like Chrome has a bug in their implementation of RTCEncodedVideoFrame and RTCEncodedAudioFrame in that they don’t define a %Symbol.toStringTag% property.

---

I’d also prefer to get this merged sooner rather than later.

[foolip]

@ExE-Boss there have been a lot of changes around idlharness.js recently, so I rebased this to see what the current state looks like.

[foolip]

Safari results look good, the only difference is in a test that I know is flaky:
https://wpt.fyi/results/media-source/idlharness.window.html?diff&filter=ADC&run_id=5742746000162816&run_id=5660754604720128

[foolip]

If the above check of a Symbol.toStringTag own property is done, then I think this whole bit can just be removed. The presence of a Symbol.toStringTag doesn't depend on whether there's a stringifier, and can be tested even in the presence of a stringifier. We don't need to also test that String() is implemented correctly, at least not for the prototype object.

[Ms2ger]

Testing @@toStringTag is great, but I think we should also make sure to test something closer to how it is used in practice, such as String().

[foolip]

Looks like Chrome has a bug in their implementation of RTCEncodedVideoFrame and RTCEncodedAudioFrame in that they don’t define a %Symbol.toStringTag% property.

I tested this a bit. Object.getOwnPropertyDescriptor(RTCEncodedVideoFrame.prototype, Symbol.toStringTag) and Object.getOwnPropertyDescriptor(RTCEncodedAudioFrame.prototype, Symbol.toStringTag) both return the expected descriptors in my testing in Chrome Stable and Canary.

Rather, it's calling String(RTCEncodedVideoFrame.prototype) and String(RTCEncodedAudioFrame.prototype) that throw the TypeError, which is because these interfaces do have stringifier in Chromum's IDL, leading to the equivalent of RTCEncodedAudioFrame.prototype.toString.call(RTCEncodedAudioFrame.prototype), which of course will throw like any attempt to invoke a method on something that isn't an instance of the interface it's on.

This is a bug in Chrome, but the way it should be detected is by checking for the absence of "toString" if there's no stringifier.

But if the final shape of this PR still has a test failing for this reason and it's kind of obscure why, that's OK, I'll file a bug about it then.

I’d also prefer to get this merged sooner rather than later.

Sorry 😳