Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ews-build] Split iOS layout test queue into no-WPT / only-WPT configurations #12508

Conversation

ryanhaddad
Copy link
Contributor

@ryanhaddad ryanhaddad commented Apr 7, 2023

3b1f649

[ews-build] Split iOS layout test queue into no-WPT / only-WPT configurations
https://bugs.webkit.org/show_bug.cgi?id=254559
rdar://107291888

Reviewed by Jonathan Bedard.

The first iteration of this commit used new steps and factories to accomplish the
no-WPT / only-WPT split, but if retries were required to determine if a failure was
pre-existing the bots ended up running the whole suite.

Using the additional arguments approach instead will make sure the no-WPT / only-WPT queues
behave as expected even in the retry case. These specific additional arguments will be
removed in the RunWebKitTestsInStressMode so they don't interfere with the intent of this
step running only the new or modified tests.

* Tools/CISupport/ews-build/config.json:
* Tools/CISupport/ews-build/factories.py:
(iOSTestsFactory):
(iOSTestsNoWPTFactory): Deleted.
(iOSTestsOnlyWPTFactory): Deleted.
* Tools/CISupport/ews-build/factories_unittest.py:
(TestExpectedBuildSteps):
* Tools/CISupport/ews-build/loadConfig.py:
* Tools/CISupport/ews-build/steps.py:
(RunWebKitTestsInStressMode):
(RunWebKitTestsNoWPT): Deleted.
(RunWebKitTestsNoWPT.setLayoutTestCommand): Deleted.
(RunWebKitTestsOnlyWPT): Deleted.
(RunWebKitTestsOnlyWPT.setLayoutTestCommand): Deleted.
* Tools/CISupport/ews-build/steps_unittest.py:
(TestRunWebKitTestsInStressMode.test_success_additional_arguments): Add new unit test.
(TestRunWebKitTestsNoWPT): Deleted.
(TestRunWebKitTestsNoWPT.setUp): Deleted.
(TestRunWebKitTestsNoWPT.tearDown): Deleted.
(TestRunWebKitTestsNoWPT.configureStep): Deleted.
(TestRunWebKitTestsNoWPT.test_success): Deleted.
(TestRunWebKitTestsOnlyWPT): Deleted.
(TestRunWebKitTestsOnlyWPT.setUp): Deleted.
(TestRunWebKitTestsOnlyWPT.tearDown): Deleted.
(TestRunWebKitTestsOnlyWPT.configureStep): Deleted.
(TestRunWebKitTestsOnlyWPT.test_success): Deleted.

Canonical link: https://commits.webkit.org/262714@main

0a8e61b

Misc iOS, tvOS & watchOS macOS Linux Windows
βœ… πŸ§ͺ style   πŸ›  ios βœ… πŸ›  mac   πŸ›  wpe βœ… πŸ›  wincairo
βœ… πŸ§ͺ bindings βœ… πŸ›  ios-sim βœ… πŸ›  mac-AS-debug   πŸ§ͺ wpe-wk2
βœ… πŸ§ͺ webkitperl   πŸ§ͺ ios-wk2   πŸ§ͺ api-mac βœ… πŸ›  gtk
  πŸ§ͺ ios-wk2-wpt   πŸ§ͺ mac-wk1   πŸ§ͺ gtk-wk2
βœ… πŸ›  tv   πŸ§ͺ api-ios   πŸ§ͺ api-gtk
  πŸ§ͺ mac-wk2   πŸ§ͺ mac-AS-debug-wk2
βœ… πŸ§ͺ services   πŸ›  tv-sim   πŸ§ͺ mac-wk2-stress
βœ… πŸ›  watch
βœ… πŸ›  πŸ§ͺ unsafe-merge   πŸ›  watch-sim

@ryanhaddad ryanhaddad self-assigned this Apr 7, 2023
@ryanhaddad ryanhaddad added the Tools / Tests Tools in the Tools directory, build issues, test infrastructure, and bugs in test cases label Apr 7, 2023
@ryanhaddad ryanhaddad force-pushed the eng/ews-build-Split-iOS-layout-test-queue-into-no-WPT--only-WPT-configurations branch from 60c70c4 to 31ff2dd Compare April 7, 2023 16:03
@ryanhaddad
Copy link
Contributor Author

The thing I'm concerned about is the conditional RunWebKitTestsInStressMode step, I think these additionalArguments will end up applying there too.

@ryanhaddad ryanhaddad force-pushed the eng/ews-build-Split-iOS-layout-test-queue-into-no-WPT--only-WPT-configurations branch from 31ff2dd to b6c9262 Compare April 7, 2023 16:30
@ryanhaddad ryanhaddad force-pushed the eng/ews-build-Split-iOS-layout-test-queue-into-no-WPT--only-WPT-configurations branch from b6c9262 to 0a8e61b Compare April 7, 2023 16:42
@ryanhaddad
Copy link
Contributor Author

ryanhaddad commented Apr 7, 2023

The thing I'm concerned about is the conditional RunWebKitTestsInStressMode step, I think these additionalArguments will end up applying there too.

Added logic to remove these arguments for this step.

@ryanhaddad ryanhaddad added skip-ews Applied to prevent a change from being run on EWS unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing labels Apr 7, 2023
…urations

https://bugs.webkit.org/show_bug.cgi?id=254559
rdar://107291888

Reviewed by Jonathan Bedard.

The first iteration of this commit used new steps and factories to accomplish the
no-WPT / only-WPT split, but if retries were required to determine if a failure was
pre-existing the bots ended up running the whole suite.

Using the additional arguments approach instead will make sure the no-WPT / only-WPT queues
behave as expected even in the retry case. These specific additional arguments will be
removed in the RunWebKitTestsInStressMode so they don't interfere with the intent of this
step running only the new or modified tests.

* Tools/CISupport/ews-build/config.json:
* Tools/CISupport/ews-build/factories.py:
(iOSTestsFactory):
(iOSTestsNoWPTFactory): Deleted.
(iOSTestsOnlyWPTFactory): Deleted.
* Tools/CISupport/ews-build/factories_unittest.py:
(TestExpectedBuildSteps):
* Tools/CISupport/ews-build/loadConfig.py:
* Tools/CISupport/ews-build/steps.py:
(RunWebKitTestsInStressMode):
(RunWebKitTestsNoWPT): Deleted.
(RunWebKitTestsNoWPT.setLayoutTestCommand): Deleted.
(RunWebKitTestsOnlyWPT): Deleted.
(RunWebKitTestsOnlyWPT.setLayoutTestCommand): Deleted.
* Tools/CISupport/ews-build/steps_unittest.py:
(TestRunWebKitTestsInStressMode.test_success_additional_arguments): Add new unit test.
(TestRunWebKitTestsNoWPT): Deleted.
(TestRunWebKitTestsNoWPT.setUp): Deleted.
(TestRunWebKitTestsNoWPT.tearDown): Deleted.
(TestRunWebKitTestsNoWPT.configureStep): Deleted.
(TestRunWebKitTestsNoWPT.test_success): Deleted.
(TestRunWebKitTestsOnlyWPT): Deleted.
(TestRunWebKitTestsOnlyWPT.setUp): Deleted.
(TestRunWebKitTestsOnlyWPT.tearDown): Deleted.
(TestRunWebKitTestsOnlyWPT.configureStep): Deleted.
(TestRunWebKitTestsOnlyWPT.test_success): Deleted.

Canonical link: https://commits.webkit.org/262714@main
@webkit-commit-queue webkit-commit-queue force-pushed the eng/ews-build-Split-iOS-layout-test-queue-into-no-WPT--only-WPT-configurations branch from 0a8e61b to 3b1f649 Compare April 7, 2023 16:59
@webkit-commit-queue
Copy link
Collaborator

Committed 262714@main (3b1f649): https://commits.webkit.org/262714@main

Reviewed commits have been landed. Closing PR #12508 and removing active labels.

@webkit-commit-queue webkit-commit-queue merged commit 3b1f649 into WebKit:main Apr 7, 2023
@webkit-commit-queue webkit-commit-queue removed the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-ews Applied to prevent a change from being run on EWS Tools / Tests Tools in the Tools directory, build issues, test infrastructure, and bugs in test cases
Projects
None yet
4 participants