Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge ios-wk2 LayoutTests/platform folder down into ios. #26364

Conversation

bls1999
Copy link
Contributor

@bls1999 bls1999 commented Mar 23, 2024

53f3bab

Merge ios-wk2 LayoutTests/platform folder down into ios.
https://bugs.webkit.org/show_bug.cgi?id=271513
rdar://125279368

Reviewed by Jonathan Bedard.

For the iOS platform, we currently only run layout tests in WebKit 2 on iOS simulator.
In order to disambiguate our layout test platform folder organization and avoid needless
distinction without a difference with regards to our current testing configurations, we
would like to merge ios-wk2 into ios in our LayoutTests/platform directory.

Expectations removed due to identical ones already present:
* accessibility/url-test.html [ Pass ]
* fast/forms/switch/pointer-tracking-there-and-back-again-rtl.html [ ImageOnlyFailure ]
* fast/forms/switch/pointer-tracking-there-and-back-again.html [ ImageOnlyFailure ]
* fast/forms/switch/pointer-tracking.html [ ImageOnlyFailure ]
* media/media-source/media-managedmse-airplay.html [ Pass ]
* media/media-source/media-managedmse-idl.html [ Pass ]
* http/tests/misc/favicon-as-image.html [ Failure ]
* http/tests/misc/iframe404.html [ Failure ]
* http/tests/misc/slow-loading-image-in-pattern.html [ Failure ]
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects.html [ Pass ]
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests.html [ Pass ]
* http/tests/app-privacy-report/ [ Pass ]
* fast/forms/validation-message-appearance.html [ Skip ]
* fast/forms/validation-message-clone.html [ Skip ]
* scrollbars/scrolling-by-page-on-keyboard-spacebar.html [ Failure ]
* http/tests/uri/css-href.py [ Failure ]
* fast/events/focus-change-crash2.html [ Skip ]
* webkit.org/b/145432 media/video-transformed-by-javascript.html [ Failure ]
* webkit.org/b/160367 media/track/track-remove-crash.html [ Skip ]
* webkit.org/b/231337 [ Debug ] imported/w3c/web-platform-tests/xhr/send-timeout-events.htm [ Pass Failure ]
* webkit.org/b/184783 compositing/ios/overflow-scroll-touch-tiles.html [ Pass Failure ]
* webkit.org/b/205216 imported/w3c/web-platform-tests/content-security-policy/reporting/report-same-origin-with-cookies.html [ Pass Failure DumpJSConsoleLogInStdErr ]
* webkit.org/b/210046 [ Release ] storage/indexeddb/value-cursor-cycle.html [ Pass Failure ]
* webkit.org/b/237295 fast/mediastream/video-rotation-gpu-process-crash.html [ Pass Crash ]
* webkit.org/b/271384 imported/w3c/web-platform-tests/css/css-ruby/line-spacing.html [ Failure ]

Expectations removed because they're outdated/incorrect (e.g., feature enabled, cannot reproduce):
* Depends on iOS 11 (new bug for failures: webkit.org/b/271778):
    * fast/viewport/ios/viewport-fit-contain.html [ Skip ]
    * fast/viewport/ios/viewport-fit-cover.html [ Skip ]
    * fast/viewport/ios/viewport-fit-auto.html [ Skip ]
* Depends on iOS 14:
    * accessibility/ios-simulator/image-overlay-elements.html [ Skip ]
* Fixed in 275423@main (webkit.org/b/267934), but is now flakily timing out (webkit.org/b/271780):
    * [ Debug ] fast/dom/Orientation/no-orientation-change-event-when-unparenting-view.html [ Crash ]
* Cannot repro, more accurate failure expectations set later (rdar://17349058):
    * fast/dom/Window/mozilla-focus-blur.html
    * fast/dom/Window/window-property-invalid-characters-ignored.html
    * fast/events/show-modal-dialog-onblur-onfocus.html
    * fast/harness/show-modal-dialog.html # new bug: webkit.org/b/271781
* Cannot repro the crash (rdar://19270614):
    * fast/doctypes/xhtml-with-xhtmlmp-doctype.xhtml [ Skip ]
    * fast/doctypes/xhtml-with-xhtmlmp11-doctype.xhtml [ Skip ]
    * fast/doctypes/xhtml-with-xhtmlmp12-doctype.xhtml [ Skip ]
    * fast/viewport/viewport-90.html [ Skip ]
    * fast/viewport/viewport-legacy-xhtmlmp-ordering.html [ Skip ]
* ^, but now constantly failing (webkit.org/b/271784):
    * fast/viewport/viewport-legacy-xhtmlmp-remove-and-add.html [ Skip ]
    * fast/viewport/viewport-legacy-xhtmlmp.html [ Skip ]
* Passing when re-enabled (webkit.org/b/231187):
    * media/modern-media-controls/overflow-support/button-state.html [ Skip ]
    * media/modern-media-controls/tracks-support/text-track-selected-via-media-api.html [ Skip ]
    * media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Skip ]
    * media/modern-media-controls/tracks-support/show-contextmenu-then-double-click-on-tracks-button.html [ Skip ]
    * media/modern-media-controls/tracks-support/off-text-track.html [ Skip ]
    * media/modern-media-controls/tracks-support/hidden-tracks.html [ Skip ]
    * media/modern-media-controls/tracks-support/click-track-in-contextmenu.html [ Skip ]
    * media/modern-media-controls/tracks-support/auto-text-track.html [ Skip ]
    * media/modern-media-controls/tracks-support/audio-multiple-tracks.html [ Skip ]
    * media/modern-media-controls/tracks-support/audio-single-track.html [ Skip ]
* ^, but now constantly timing out (webkit.org/b/271785):
    * media/modern-media-controls/overflow-support/playback-speed.html [ Skip ]
    * media/modern-media-controls/overflow-support/chapters.html [ Skip ]
* Newer expectations for ImageOnlyFailure set later in the file (webkit.org/b/214299):
    * imported/w3c/web-platform-tests/css/css-ui/appearance-menulist-button-002.tentative.html [ Pass ]
    * imported/w3c/web-platform-tests/css/css-ui/webkit-appearance-menulist-button-002.tentative.html [ Pass ]
* Last "something other than pass"ed in iOS 16:
    * imported/blink/compositing/video/video-controls-layer-creation-squashing.html [ Pass Timeout Crash ] # webkit.org/b/232412
    * imported/blink/compositing/video/video-controls-layer-creation-squashing.html [ ImageOnlyFailure ]
* Last failed in October 2023 (webkit.org/b/261234):
    * fast/forms/auto-fill-button/caps-lock-indicator-should-be-visible-after-hiding-auto-fill-strong-password-button.html [ Pass ImageOnlyFailure ]
* Last failed in March 2023 (webkit.org/b/207647):
    * http/tests/security/strip-referrer-to-origin-for-third-party-redirects-in-private-mode.html [ Pass Timeout ]
* More recent/accurate expectation already set:
    * scrollingcoordinator/ios/fixed-frame-overflow-swipe.html [ ImageOnlyFailure ]
* Incorrect (rdar://106007933):
    * editing/selection/ios/changing-selection-does-not-trigger-autocapitalization.html [ Failure ]
* Depends on iOS 17 (rdar://103646683):
    * editing/selection/ios/show-grammar-replacements-on-tap.html [ Skip ]
    * editing/selection/ios/show-grammar-replacements-on-tap.html [ Pass ]
* Cannot repro any failures (webkit.org/b/182850):
    * editing/pasteboard/data-transfer-set-data-sanitizes-html-when-copying-in-null-origin.html [ Skip ]
* Stale bugs:
    * webkit.org/b/184787 http/wpt/cache-storage/cache-quota.any.html [ Pass Failure ]
    * webkit.org/b/217412 http/wpt/cache-storage/cache-quota.any.html [ Pass Failure ]
* Not supported in simulator (expectations redundant):
    * http/tests/media/fairplay [ Pass ]
    * http/tests/media/fairplay [ Skip ]

Rebaselined (failing on EWS/locally):
* compositing/iframes/compositing-for-scrollable-iframe.html
* css2.1/t0803-c5502-mrgn-r-03-c.html
* css2.1/t0803-c5504-mrgn-l-03-c.html
* css2.1/t0804-c5507-padn-r-03-f.html
* css2.1/t0804-c5510-padn-02-f.html
* css2.1/t0905-c5525-flthw-00-c-g.html
* css2.1/t0905-c5526-flthw-00-c-g.html
* css2.1/t1601-c547-indent-00-b-a.html
* css2.1/t1604-c542-letter-sp-00-b-a.html
* css2.1/t040302-c61-rel-len-00-b-ag.html
* css2.1/t100801-c548-ln-ht-02-b-ag.html
* fast/body-propagation/overflow/005-declarative.xhtml
* fast/body-propagation/overflow/005-xhtml.xhtml
* fast/body-propagation/overflow/005.html
* fast/body-propagation/overflow/006-declarative.xhtml
* fast/body-propagation/overflow/006-xhtml.xhtml
* fast/body-propagation/overflow/006.html
* fast/forms/textarea-placeholder-visibility-1.html
* fast/forms/textarea-placeholder-visibility-2.html
* fast/hidpi/resize-corner-hidpi.html
* fast/overflow/overflow-with-local-background-attachment.html

File changes:
* LayoutTests/platform/ios-wk2/TestExpectations: Removed.
* LayoutTests/platform/ios/TestExpectations: Merged test expectations from ios-wk2 into ios.
* LayoutTests/platform/ios/*/: Merged test baselines from ios-wk2 into ios.

Canonical link: https://commits.webkit.org/276785@main

e4c3470

Misc iOS, tvOS & watchOS macOS Linux Windows
❌ πŸ§ͺ style βœ… πŸ›  ios βœ… πŸ›  mac   πŸ›  wpe   πŸ›  wincairo
βœ… πŸ›  ios-sim βœ… πŸ›  mac-AS-debug   πŸ§ͺ wpe-wk2
❌ πŸ§ͺ webkitperl   πŸ§ͺ ios-wk2   πŸ§ͺ api-mac   πŸ§ͺ api-wpe
  πŸ§ͺ ios-wk2-wpt   πŸ§ͺ mac-wk1 βœ… πŸ›  wpe-skia
  πŸ§ͺ api-ios   πŸ§ͺ mac-wk2   πŸ›  gtk
  πŸ›  tv   πŸ§ͺ mac-AS-debug-wk2   πŸ§ͺ gtk-wk2
  πŸ›  tv-sim   πŸ§ͺ api-gtk
  πŸ›  watch
βœ… πŸ›  πŸ§ͺ unsafe-merge   πŸ›  watch-sim

@bls1999 bls1999 self-assigned this Mar 23, 2024
@bls1999 bls1999 added the Tools / Tests Tools in the Tools directory, build issues, test infrastructure, and bugs in test cases label Mar 23, 2024
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 23, 2024
@bls1999 bls1999 removed the merging-blocked Applied to prevent a change from being merged label Mar 27, 2024
@bls1999 bls1999 force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from 8d1c346 to 1dd0e72 Compare March 27, 2024 20:28
@bls1999 bls1999 force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from 1dd0e72 to 4c5fc6f Compare March 27, 2024 20:54
@bls1999 bls1999 force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from 4c5fc6f to 2e5d062 Compare March 28, 2024 01:43
@bls1999 bls1999 requested a review from JonWBedard March 28, 2024 02:47
@bls1999 bls1999 marked this pull request as ready for review March 28, 2024 02:47
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 28, 2024
@bls1999 bls1999 removed the merging-blocked Applied to prevent a change from being merged label Mar 28, 2024
@bls1999 bls1999 force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from 2e5d062 to ec32e7b Compare March 28, 2024 04:36
@bls1999 bls1999 force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from ec32e7b to e4c3470 Compare March 28, 2024 17:23
@bls1999 bls1999 added the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 28, 2024
https://bugs.webkit.org/show_bug.cgi?id=271513
rdar://125279368

Reviewed by Jonathan Bedard.

For the iOS platform, we currently only run layout tests in WebKit 2 on iOS simulator.
In order to disambiguate our layout test platform folder organization and avoid needless
distinction without a difference with regards to our current testing configurations, we
would like to merge ios-wk2 into ios in our LayoutTests/platform directory.

Expectations removed due to identical ones already present:
* accessibility/url-test.html [ Pass ]
* fast/forms/switch/pointer-tracking-there-and-back-again-rtl.html [ ImageOnlyFailure ]
* fast/forms/switch/pointer-tracking-there-and-back-again.html [ ImageOnlyFailure ]
* fast/forms/switch/pointer-tracking.html [ ImageOnlyFailure ]
* media/media-source/media-managedmse-airplay.html [ Pass ]
* media/media-source/media-managedmse-idl.html [ Pass ]
* http/tests/misc/favicon-as-image.html [ Failure ]
* http/tests/misc/iframe404.html [ Failure ]
* http/tests/misc/slow-loading-image-in-pattern.html [ Failure ]
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-redirects.html [ Pass ]
* http/tests/resourceLoadStatistics/strip-referrer-to-origin-for-third-party-requests.html [ Pass ]
* http/tests/app-privacy-report/ [ Pass ]
* fast/forms/validation-message-appearance.html [ Skip ]
* fast/forms/validation-message-clone.html [ Skip ]
* scrollbars/scrolling-by-page-on-keyboard-spacebar.html [ Failure ]
* http/tests/uri/css-href.py [ Failure ]
* fast/events/focus-change-crash2.html [ Skip ]
* webkit.org/b/145432 media/video-transformed-by-javascript.html [ Failure ]
* webkit.org/b/160367 media/track/track-remove-crash.html [ Skip ]
* webkit.org/b/231337 [ Debug ] imported/w3c/web-platform-tests/xhr/send-timeout-events.htm [ Pass Failure ]
* webkit.org/b/184783 compositing/ios/overflow-scroll-touch-tiles.html [ Pass Failure ]
* webkit.org/b/205216 imported/w3c/web-platform-tests/content-security-policy/reporting/report-same-origin-with-cookies.html [ Pass Failure DumpJSConsoleLogInStdErr ]
* webkit.org/b/210046 [ Release ] storage/indexeddb/value-cursor-cycle.html [ Pass Failure ]
* webkit.org/b/237295 fast/mediastream/video-rotation-gpu-process-crash.html [ Pass Crash ]
* webkit.org/b/271384 imported/w3c/web-platform-tests/css/css-ruby/line-spacing.html [ Failure ]

Expectations removed because they're outdated/incorrect (e.g., feature enabled, cannot reproduce):
* Depends on iOS 11 (new bug for failures: webkit.org/b/271778):
    * fast/viewport/ios/viewport-fit-contain.html [ Skip ]
    * fast/viewport/ios/viewport-fit-cover.html [ Skip ]
    * fast/viewport/ios/viewport-fit-auto.html [ Skip ]
* Depends on iOS 14:
    * accessibility/ios-simulator/image-overlay-elements.html [ Skip ]
* Fixed in 275423@main (webkit.org/b/267934), but is now flakily timing out (webkit.org/b/271780):
    * [ Debug ] fast/dom/Orientation/no-orientation-change-event-when-unparenting-view.html [ Crash ]
* Cannot repro, more accurate failure expectations set later (rdar://17349058):
    * fast/dom/Window/mozilla-focus-blur.html
    * fast/dom/Window/window-property-invalid-characters-ignored.html
    * fast/events/show-modal-dialog-onblur-onfocus.html
    * fast/harness/show-modal-dialog.html # new bug: webkit.org/b/271781
* Cannot repro the crash (rdar://19270614):
    * fast/doctypes/xhtml-with-xhtmlmp-doctype.xhtml [ Skip ]
    * fast/doctypes/xhtml-with-xhtmlmp11-doctype.xhtml [ Skip ]
    * fast/doctypes/xhtml-with-xhtmlmp12-doctype.xhtml [ Skip ]
    * fast/viewport/viewport-90.html [ Skip ]
    * fast/viewport/viewport-legacy-xhtmlmp-ordering.html [ Skip ]
* ^, but now constantly failing (webkit.org/b/271784):
    * fast/viewport/viewport-legacy-xhtmlmp-remove-and-add.html [ Skip ]
    * fast/viewport/viewport-legacy-xhtmlmp.html [ Skip ]
* Passing when re-enabled (webkit.org/b/231187):
    * media/modern-media-controls/overflow-support/button-state.html [ Skip ]
    * media/modern-media-controls/tracks-support/text-track-selected-via-media-api.html [ Skip ]
    * media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Skip ]
    * media/modern-media-controls/tracks-support/show-contextmenu-then-double-click-on-tracks-button.html [ Skip ]
    * media/modern-media-controls/tracks-support/off-text-track.html [ Skip ]
    * media/modern-media-controls/tracks-support/hidden-tracks.html [ Skip ]
    * media/modern-media-controls/tracks-support/click-track-in-contextmenu.html [ Skip ]
    * media/modern-media-controls/tracks-support/auto-text-track.html [ Skip ]
    * media/modern-media-controls/tracks-support/audio-multiple-tracks.html [ Skip ]
    * media/modern-media-controls/tracks-support/audio-single-track.html [ Skip ]
* ^, but now constantly timing out (webkit.org/b/271785):
    * media/modern-media-controls/overflow-support/playback-speed.html [ Skip ]
    * media/modern-media-controls/overflow-support/chapters.html [ Skip ]
* Newer expectations for ImageOnlyFailure set later in the file (webkit.org/b/214299):
    * imported/w3c/web-platform-tests/css/css-ui/appearance-menulist-button-002.tentative.html [ Pass ]
    * imported/w3c/web-platform-tests/css/css-ui/webkit-appearance-menulist-button-002.tentative.html [ Pass ]
* Last "something other than pass"ed in iOS 16:
    * imported/blink/compositing/video/video-controls-layer-creation-squashing.html [ Pass Timeout Crash ] # webkit.org/b/232412
    * imported/blink/compositing/video/video-controls-layer-creation-squashing.html [ ImageOnlyFailure ]
* Last failed in October 2023 (webkit.org/b/261234):
    * fast/forms/auto-fill-button/caps-lock-indicator-should-be-visible-after-hiding-auto-fill-strong-password-button.html [ Pass ImageOnlyFailure ]
* Last failed in March 2023 (webkit.org/b/207647):
    * http/tests/security/strip-referrer-to-origin-for-third-party-redirects-in-private-mode.html [ Pass Timeout ]
* More recent/accurate expectation already set:
    * scrollingcoordinator/ios/fixed-frame-overflow-swipe.html [ ImageOnlyFailure ]
* Incorrect (rdar://106007933):
    * editing/selection/ios/changing-selection-does-not-trigger-autocapitalization.html [ Failure ]
* Depends on iOS 17 (rdar://103646683):
    * editing/selection/ios/show-grammar-replacements-on-tap.html [ Skip ]
    * editing/selection/ios/show-grammar-replacements-on-tap.html [ Pass ]
* Cannot repro any failures (webkit.org/b/182850):
    * editing/pasteboard/data-transfer-set-data-sanitizes-html-when-copying-in-null-origin.html [ Skip ]
* Stale bugs:
    * webkit.org/b/184787 http/wpt/cache-storage/cache-quota.any.html [ Pass Failure ]
    * webkit.org/b/217412 http/wpt/cache-storage/cache-quota.any.html [ Pass Failure ]
* Not supported in simulator (expectations redundant):
    * http/tests/media/fairplay [ Pass ]
    * http/tests/media/fairplay [ Skip ]

Rebaselined (failing on EWS/locally):
* compositing/iframes/compositing-for-scrollable-iframe.html
* css2.1/t0803-c5502-mrgn-r-03-c.html
* css2.1/t0803-c5504-mrgn-l-03-c.html
* css2.1/t0804-c5507-padn-r-03-f.html
* css2.1/t0804-c5510-padn-02-f.html
* css2.1/t0905-c5525-flthw-00-c-g.html
* css2.1/t0905-c5526-flthw-00-c-g.html
* css2.1/t1601-c547-indent-00-b-a.html
* css2.1/t1604-c542-letter-sp-00-b-a.html
* css2.1/t040302-c61-rel-len-00-b-ag.html
* css2.1/t100801-c548-ln-ht-02-b-ag.html
* fast/body-propagation/overflow/005-declarative.xhtml
* fast/body-propagation/overflow/005-xhtml.xhtml
* fast/body-propagation/overflow/005.html
* fast/body-propagation/overflow/006-declarative.xhtml
* fast/body-propagation/overflow/006-xhtml.xhtml
* fast/body-propagation/overflow/006.html
* fast/forms/textarea-placeholder-visibility-1.html
* fast/forms/textarea-placeholder-visibility-2.html
* fast/hidpi/resize-corner-hidpi.html
* fast/overflow/overflow-with-local-background-attachment.html

File changes:
* LayoutTests/platform/ios-wk2/TestExpectations: Removed.
* LayoutTests/platform/ios/TestExpectations: Merged test expectations from ios-wk2 into ios.
* LayoutTests/platform/ios/*/: Merged test baselines from ios-wk2 into ios.

Canonical link: https://commits.webkit.org/276785@main
@webkit-commit-queue webkit-commit-queue force-pushed the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch from e4c3470 to 53f3bab Compare March 28, 2024 17:26
@webkit-commit-queue
Copy link
Collaborator

Committed 276785@main (53f3bab): https://commits.webkit.org/276785@main

Reviewed commits have been landed. Closing PR #26364 and removing active labels.

@webkit-commit-queue webkit-commit-queue merged commit 53f3bab into WebKit:main Mar 28, 2024
@webkit-commit-queue webkit-commit-queue removed the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 28, 2024
@bls1999 bls1999 deleted the eng/Merge-ios-wk2-LayoutTestsplatform-folder-down-into-ios- branch March 29, 2024 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Tools / Tests Tools in the Tools directory, build issues, test infrastructure, and bugs in test cases
Projects
None yet
5 participants