Chrome Dev & Edge Dev

[jensimmons]

I just ran across yet another page on Can I Use where Chrome and Edge have different support for a thing. (One supports it, the other doesn't.) Now that Can I Use put Chrome and Edge desktop browsers next to each other, it's very easy to see such differences jump out.

Which led me to wonder — why do they have the same column on the Interop dashboard? Are we in fact testing both? Or are we testing Chrome, and just labeling that score as "Edge, too. Must be the same."

[foolip]

For https://wpt.fyi/interop-2022 we only use the results from Chrome and assume that the results for Edge will be the same. Because they don't update at the exact same time there will definitely be differences at times however. Here's a link with a diff for all of Interop 2022:

https://wpt.fyi/results/?label=master&label=experimental&product=chrome&product=edge&aligned&diff&filter=ADC&q=label%3Ainterop-2021-aspect-ratio%20or%20label%3Ainterop-2021-flexbox%20or%20label%3Ainterop-2021-grid%20or%20label%3Ainterop-2021-position-sticky%20or%20label%3Ainterop-2021-transforms%20or%20label%3Ainterop-2022-cascade%20or%20label%3Ainterop-2022-color%20or%20label%3Ainterop-2022-contain%20or%20label%3Ainterop-2022-dialog%20or%20label%3Ainterop-2022-forms%20or%20label%3Ainterop-2022-scrolling%20or%20label%3Ainterop-2022-subgrid%20or%20label%3Ainterop-2022-text%20or%20label%3Ainterop-2022-viewport%20or%20label%3Ainterop-2022-webcompat

There are some Edge failures like this one:
https://wpt.fyi/results/html/semantics/interactive-elements/the-dialog-element/modal-dialog-in-replaced-renderer.html?diff&filter=ADC&run_id=5762866111053824&run_id=5668150002581504

But most of this seems to be missing results in Edge. I'll file an issue about this.

Note that it would be technically straightforward to show Edge and Chrome separately, but past feedback on wpt.fyi like web-platform-tests/wpt.fyi#1519 made us avoid that for Compat 2021 and that choice then carried forward.

[foolip]

When it comes to differences between Chrome and Edge in Can I Use or BCD, most such differences are probably incorrect, at least in BCD. Improving on that is part of openwebdocs/project#119, in particular openwebdocs/project#85.

[foolip]

I've filed web-platform-tests/wpt#34913.

[gsnedders]

This implies there's some differences: https://wpt.fyi/results/?label=master&label=experimental&aligned&q=%28label%3Ainterop-2021-aspect-ratio%20or%20label%3Ainterop-2021-flexbox%20or%20label%3Ainterop-2021-grid%20or%20label%3Ainterop-2021-position-sticky%20or%20label%3Ainterop-2021-transforms%20or%20label%3Ainterop-2022-cascade%20or%20label%3Ainterop-2022-color%20or%20label%3Ainterop-2022-contain%20or%20label%3Ainterop-2022-dialog%20or%20label%3Ainterop-2022-forms%20or%20label%3Ainterop-2022-scrolling%20or%20label%3Ainterop-2022-subgrid%20or%20label%3Ainterop-2022-typography%20or%20label%3Ainterop-2022-viewport%20or%20label%3Ainterop-2022-webcompat%29%20count%3A1%28chrome%3Apass%20or%20edge%3Apass%20or%20chrome%3Aok%20or%20edge%3Aok%29

[foolip]

Yes, there certainly are differences, some might be persistent even and wouldn't go away with time.

@dlibby- @kypflug do you have thoughts on our current setup?

Also cc @jgraham given web-platform-tests/wpt.fyi#1519.

[dlibby-]

I'm not very familiar with the setup, but sampling the failures, I see these broad categories:

• Incomplete results (looks tracked by Incomplete Edge Dev results wpt#34913, Mustapha is the best person to comment on those)
• Legitimate differences (i.e. the modal dialog tests, not sure why we have a difference there, but filed an internal bug for the owning team to follow up)
• Anti-aliasing pixel differences.
• Successive Ahem 'X' characters aren't creating a solid rectangle in some of the tests.

For the last two I can reproduce the same results in Chrome on Windows. The anti-aliasing differences I spot checked are a bit odd, given that these are all reftests, AFAICT.

Given that Edge is the only Windows-based run, it's possible that there are platform-specific differences or bugs for Chromium that are causing these failures. The interesting bit to me is that we have these tests running in both Chromium and Edge CI and change validation and don't see the same failures on Windows. So perhaps there's a test configuration (intuition says something related turning off font anti-aliasing) that causes these to show up.

I'm going to be out for the next few days, but I'll follow up with more detailed investigation and finish combing through the failures next week.

[foolip]

This has been resolved in https://wpt.fyi/interop-2024 by separating Chrome and Edge.