[API Tests] Reduce run-api-tests startup overhead when running specific tests by rr-codes · Pull Request #60911 · WebKit/WebKit

rr-codes · 2026-03-19T04:03:45Z

`a80da14`

[API Tests] Reduce run-api-tests startup overhead when running specific tests
https://bugs.webkit.org/show_bug.cgi?id=310247
rdar://172887928

Reviewed by Jonathan Bedard.

This patch reduces the startup overhead of run-api-tests by addressing
several bottlenecks in the Python script's initialization path.

1. Defer default_configuration() in configuration_options() (~500ms)

The -t/--target option eagerly created a Config object and called
default_configuration(), which spawns a Perl subprocess
(webkit-build-directory) to determine the build directory. This
happened unconditionally at option-parse time, even when --debug or
--release was explicitly passed. Changed the default to None; the
port base class (base.py:168-169) already has a lazy fallback that
calls default_configuration() only when no configuration was set.

2. Make SDK resolution lazy in embedded port modules (~1-1.5s)

Six embedded port modules (ios_simulator, ios_device,
watch_simulator, watch_device, visionos_simulator, visionos_device)
called apple_additions().get_sdk() at class-definition time, which
triggered xcodebuild -showsdks (~1s+) the moment any of these
modules was imported by PortFactory.get(). Fixed by converting SDK
from a class-level attribute to a lazy @property that defers the
lookup until first access. Each class now has a _DEFAULT_SDK class
attribute for the base SDK name and a @property SDK that calls
apple_additions().get_sdk() only when needed. This aligns with the
EmbeddedPort base class, which already declares SDK as
@property @abstractmethod. Also updated EmbeddedDevicePort's
determine_full_port_name() classmethod to use cls._DEFAULT_SDK
instead of cls.SDK, since properties don't work on class references.
Additionally, PORT_CLASSES is restored to alphabetical order and
the import logic is extracted into a _load_port_class() helper.

3. Defer configuration_for_upload until needed (~100-200ms)

configuration_for_upload was computed unconditionally before test
collection, but is only needed when --report-urls is specified.
Moved it into the upload block.

4. Skip test collection when exact test names are provided.

Previously, `_collect_tests` always copied the test binary to "ToBeListed" and
spawned it with --gtest_list_tests to enumerate every available test, even when
the user specified exact test names on the command line (~0.5-1s of overhead).

Fix by adding `_try_collect_exact_tests`, which detects when all arguments are
fully-qualified test names (no glob wildcards) and resolves them directly
without listing. For the `<Binary>.<Suite>.<Test>` form, the name is used as-is.
For the `<Suite>.<Test>` form, the appropriate binary prefix is added. The method
falls back to the full listing path for glob patterns, partial matches like
`<Binary>.<Suite>`, or when no arguments are given.

5. Remove unnecessary preference reset from API test runs.

`_set_up_run` unconditionally called `reset_preferences()`, which runs two
sequential `defaults delete` commands for the DumpRenderTree and
WebKitTestRunner preference domains (~0.1s). These are layout test runner
preference domains and are not used by the API test binaries.

6. Skip build check when --no-build is set.

`check_api_test_build` was called unconditionally, resolving binary paths and
checking filesystem existence even when --no-build (the default) was passed.
Now the entire build check is skipped when build=False, deferring the first
build-directory resolution to `_collect_tests` where it's actually needed.

7. Lazy-import webkitscmpy in port/base.py (~150-200ms)

`from webkitscmpy import local` was a module-level import that loaded the
entire webkitscmpy dependency graph (registering dozens of AutoInstall
packages) on every run. Moved it into `commits_for_upload()`, the only
method that uses it.

8. Cache path_to_api_test_binaries() in check_api_test_build.

`check_api_test_build` called `path_to_api_test_binaries()` twice: once to
get the key list and once to iterate. Stored the result in a local variable
to avoid the redundant dict reconstruction.

9. Lazy-import Upload and remove dead imports in manager.py.

Moved `from webkitpy.results.upload import Upload` into the upload codepath
where it's actually used. Removed completely unused `DeviceRequest` and
`SimulatedDeviceManager` imports.

* Tools/Scripts/webkitpy/api_tests/manager.py:
(Manager):
(Manager._is_exact_test_name):
(Manager._try_collect_exact_tests):
(Manager._collect_tests):
(Manager._set_up_run):
(Manager.run):
* Tools/Scripts/webkitpy/api_tests/manager_unittest.py:
(test_is_exact_test_name):
* Tools/Scripts/webkitpy/port/base.py:
(Port.check_api_test_build):
(Port.commits_for_upload):
* Tools/Scripts/webkitpy/port/embedded_device.py:
(EmbeddedDevicePort.determine_full_port_name):
* Tools/Scripts/webkitpy/port/factory.py:
(configuration_options):
(PortFactory):
(PortFactory._load_port_class):
(PortFactory.get):
(PortFactory.get.in):
* Tools/Scripts/webkitpy/port/ios_device.py:
(IOSDevicePort):
(IOSDevicePort.SDK):
* Tools/Scripts/webkitpy/port/ios_simulator.py:
(IOSSimulatorPort):
(IOSSimulatorPort.SDK):
* Tools/Scripts/webkitpy/port/port_testcase.py:
(bind_mock_apple_additions.MockAppleAdditions):
(bind_mock_apple_additions.MockAppleAdditions.get_sdk):
* Tools/Scripts/webkitpy/port/visionos_device.py:
(VisionOSDevicePort):
(VisionOSDevicePort.SDK):
* Tools/Scripts/webkitpy/port/visionos_simulator.py:
(VisionOSSimulatorPort):
(VisionOSSimulatorPort.SDK):
* Tools/Scripts/webkitpy/port/watch_device.py:
(WatchDevicePort):
(WatchDevicePort.SDK):
* Tools/Scripts/webkitpy/port/watch_simulator.py:
(WatchSimulatorPort):
(WatchSimulatorPort.SDK):

Canonical link: https://commits.webkit.org/309636@main

6e1cfbb

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	~~🛠 win~~	⏳ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	~~🧪 win-tests~~	🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🧪 api-wpe		✅ 🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	~~🧪 gtk-wk2~~
	✅ 🛠 vision	~~🧪 mac-AS-debug-wk2~~	✅ 🧪 api-gtk
~~🛠 🧪 merge~~	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
✅ 🛠 🧪 unsafe-merge	✅ 🧪 vision-wk2	~~🧪 mac-intel-wk2~~
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2026-03-19T04:03:51Z

EWS run on previous version of this PR (hash b9a3d91)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	~~🛠 mac~~	✅ 🛠 wpe	~~🛠 win~~	⏳ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~	~~🧪 win-tests~~	⏳ 🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	~~🧪 api-wpe~~		⏳ 🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	~~🧪 gtk-wk2~~
	~~🛠 vision~~	~~🧪 mac-AS-debug-wk2~~	~~🧪 api-gtk~~
	✅ 🛠 vision-sim	~~🧪 mac-wk2-stress~~	~~🛠 playstation~~
	~~🧪 vision-wk2~~	~~🧪 mac-intel-wk2~~
	~~🛠 tv~~
	~~🛠 tv-sim~~
	~~🛠 watch~~
	~~🛠 watch-sim~~

webkit-early-warning-system · 2026-03-19T04:16:06Z

EWS run on previous version of this PR (hash e244c7f)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	~~🛠 win~~	🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~	~~🧪 win-tests~~	✅ 🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🧪 api-wpe		❌ 🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	✅ 🧪 gtk-wk2
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
	~~🧪 vision-wk2~~	~~🧪 mac-intel-wk2~~
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2026-03-19T05:03:09Z

EWS run on previous version of this PR (hash 3b39240)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win	⏳ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	✅ 🧪 win-tests	✅ 🛠 mac-apple
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe		⏳ 🛠 vision-apple
✅ 🧪 webkitpy	✅ 🧪 ios-wk2-wpt	✅ 🧪 api-mac-debug	✅ 🛠 gtk3-libwebrtc
	✅ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🛠 gtk
		✅ 🧪 mac-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

gsnedders

I think @JonWBedard, and probably @Smackteo, should have a look at the API tests changes.

Tools/Scripts/webkitpy/port/factory.py

Tools/Scripts/webkitpy/api_tests/manager.py

webkit-early-warning-system · 2026-03-19T20:58:24Z

EWS run on previous version of this PR (hash 82a27e8)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	~~🛠 mac~~	✅ 🛠 wpe	~~🛠 win~~	✅ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~	~~🧪 win-tests~~	❌ 🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🧪 api-wpe		🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	~~🧪 gtk-wk2~~
	~~🛠 vision~~	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	~~🧪 mac-wk2-stress~~	~~🛠 playstation~~
	~~🧪 vision-wk2~~	~~🧪 mac-intel-wk2~~
	✅ 🛠 tv
	~~🛠 tv-sim~~
	~~🛠 watch~~
	~~🛠 watch-sim~~

webkit-early-warning-system · 2026-03-19T21:42:32Z

EWS run on previous version of this PR (hash 39ffd79)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	~~🛠 mac~~	✅ 🛠 wpe	~~🛠 win~~	⏳ 🛠 ios-apple
✅ 🧪 bindings	~~🛠 ios-sim~~	✅ 🛠 mac-AS-debug	~~🧪 wpe-wk2~~	~~🧪 win-tests~~	⏳ 🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🧪 api-wpe		⏳ 🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	~~🧪 gtk-wk2~~
	~~🛠 vision~~	~~🧪 mac-AS-debug-wk2~~	✅ 🧪 api-gtk
	~~🛠 vision-sim~~	~~🧪 mac-wk2-stress~~	~~🛠 playstation~~
	~~🧪 vision-wk2~~	~~🧪 mac-intel-wk2~~
	✅ 🛠 tv
	~~🛠 tv-sim~~
	~~🛠 watch~~
	~~🛠 watch-sim~~

webkit-early-warning-system · 2026-03-19T21:59:36Z

EWS run on previous version of this PR (hash 21e9eff)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	~~🛠 win~~	⏳ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	~~🧪 win-tests~~	⏳ 🛠 mac-apple
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe		🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	✅ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🛠 gtk
		✅ 🧪 mac-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	~~🛠 playstation~~
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	~~🛠 watch-sim~~

webkit-early-warning-system · 2026-03-19T23:22:16Z

EWS run on previous version of this PR (hash 5a1eba7)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win	✅ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	~~🧪 win-tests~~	⏳ 🛠 mac-apple
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe		✅ 🛠 vision-apple
✅ 🧪 webkitpy	✅ 🧪 ios-wk2-wpt	✅ 🧪 api-mac-debug	✅ 🛠 gtk3-libwebrtc
✅ 🛠 🧪 jsc	✅ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🛠 gtk
✅ 🛠 🧪 jsc-debug-arm64	✅ 🛠 ios-safer-cpp	✅ 🧪 mac-wk2	✅ 🧪 gtk-wk2
✅ 🧪 services	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2	✅ 🛠 jsc-armv7
	✅ 🛠 tv	✅ 🛠 mac-safer-cpp	✅ 🧪 jsc-armv7-tests
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2026-03-20T04:01:19Z

EWS run on previous version of this PR (hash eac26fd)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	~~🛠 win~~	🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	~~🧪 win-tests~~	✅ 🛠 mac-apple
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe		⏳ 🛠 vision-apple
✅ 🧪 webkitpy	✅ 🧪 ios-wk2-wpt	✅ 🧪 api-mac-debug	✅ 🛠 gtk3-libwebrtc
	✅ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🛠 gtk
		✅ 🧪 mac-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision	✅ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-early-warning-system · 2026-03-20T06:34:04Z

EWS run on previous version of this PR (hash 9120ab2)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 win	✅ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	❌ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	✅ 🧪 win-tests	⏳ 🛠 mac-apple
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 api-wpe		✅ 🛠 vision-apple
✅ 🧪 webkitpy	✅ 🧪 ios-wk2-wpt	❌ 🧪 api-mac-debug	✅ 🛠 gtk3-libwebrtc
	✅ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🛠 gtk
		✅ 🧪 mac-wk2	✅ 🧪 gtk-wk2
	✅ 🛠 vision	❌ 🧪 mac-AS-debug-wk2	✅ 🧪 api-gtk
❌ 🛠 🧪 merge	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
	✅ 🧪 vision-wk2	✅ 🧪 mac-intel-wk2
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

webkit-commit-queue · 2026-03-20T18:56:04Z

No reviewer information in commit message, blocking PR #60911. Details: Build #33902

webkit-early-warning-system · 2026-03-20T20:49:54Z

EWS run on current version of this PR (hash 6e1cfbb)

Details

Misc	iOS, visionOS, tvOS & watchOS	macOS	Linux	Windows	Apple Internal
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	~~🛠 win~~	⏳ 🛠 ios-apple
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🧪 wpe-wk2	~~🧪 win-tests~~	🛠 mac-apple
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	~~🧪 api-mac~~	✅ 🧪 api-wpe		✅ 🛠 vision-apple
✅ 🧪 webkitpy	~~🧪 ios-wk2-wpt~~	~~🧪 api-mac-debug~~	✅ 🛠 gtk3-libwebrtc
	~~🧪 api-ios~~	~~🧪 mac-wk1~~	✅ 🛠 gtk
		~~🧪 mac-wk2~~	~~🧪 gtk-wk2~~
	✅ 🛠 vision	~~🧪 mac-AS-debug-wk2~~	✅ 🧪 api-gtk
~~🛠 🧪 merge~~	✅ 🛠 vision-sim	✅ 🧪 mac-wk2-stress	✅ 🛠 playstation
✅ 🛠 🧪 unsafe-merge	✅ 🧪 vision-wk2	~~🧪 mac-intel-wk2~~
	✅ 🛠 tv
	✅ 🛠 tv-sim
	✅ 🛠 watch
	✅ 🛠 watch-sim

@AbstractMethod

…ic tests https://bugs.webkit.org/show_bug.cgi?id=310247 rdar://172887928 Reviewed by Jonathan Bedard. This patch reduces the startup overhead of run-api-tests by addressing several bottlenecks in the Python script's initialization path. 1. Defer default_configuration() in configuration_options() (~500ms) The -t/--target option eagerly created a Config object and called default_configuration(), which spawns a Perl subprocess (webkit-build-directory) to determine the build directory. This happened unconditionally at option-parse time, even when --debug or --release was explicitly passed. Changed the default to None; the port base class (base.py:168-169) already has a lazy fallback that calls default_configuration() only when no configuration was set. 2. Make SDK resolution lazy in embedded port modules (~1-1.5s) Six embedded port modules (ios_simulator, ios_device, watch_simulator, watch_device, visionos_simulator, visionos_device) called apple_additions().get_sdk() at class-definition time, which triggered xcodebuild -showsdks (~1s+) the moment any of these modules was imported by PortFactory.get(). Fixed by converting SDK from a class-level attribute to a lazy @Property that defers the lookup until first access. Each class now has a _DEFAULT_SDK class attribute for the base SDK name and a @Property SDK that calls apple_additions().get_sdk() only when needed. This aligns with the EmbeddedPort base class, which already declares SDK as @Property @AbstractMethod. Also updated EmbeddedDevicePort's determine_full_port_name() classmethod to use cls._DEFAULT_SDK instead of cls.SDK, since properties don't work on class references. Additionally, PORT_CLASSES is restored to alphabetical order and the import logic is extracted into a _load_port_class() helper. 3. Defer configuration_for_upload until needed (~100-200ms) configuration_for_upload was computed unconditionally before test collection, but is only needed when --report-urls is specified. Moved it into the upload block. 4. Skip test collection when exact test names are provided. Previously, `_collect_tests` always copied the test binary to "ToBeListed" and spawned it with --gtest_list_tests to enumerate every available test, even when the user specified exact test names on the command line (~0.5-1s of overhead). Fix by adding `_try_collect_exact_tests`, which detects when all arguments are fully-qualified test names (no glob wildcards) and resolves them directly without listing. For the `<Binary>.<Suite>.<Test>` form, the name is used as-is. For the `<Suite>.<Test>` form, the appropriate binary prefix is added. The method falls back to the full listing path for glob patterns, partial matches like `<Binary>.<Suite>`, or when no arguments are given. 5. Remove unnecessary preference reset from API test runs. `_set_up_run` unconditionally called `reset_preferences()`, which runs two sequential `defaults delete` commands for the DumpRenderTree and WebKitTestRunner preference domains (~0.1s). These are layout test runner preference domains and are not used by the API test binaries. 6. Skip build check when --no-build is set. `check_api_test_build` was called unconditionally, resolving binary paths and checking filesystem existence even when --no-build (the default) was passed. Now the entire build check is skipped when build=False, deferring the first build-directory resolution to `_collect_tests` where it's actually needed. 7. Lazy-import webkitscmpy in port/base.py (~150-200ms) `from webkitscmpy import local` was a module-level import that loaded the entire webkitscmpy dependency graph (registering dozens of AutoInstall packages) on every run. Moved it into `commits_for_upload()`, the only method that uses it. 8. Cache path_to_api_test_binaries() in check_api_test_build. `check_api_test_build` called `path_to_api_test_binaries()` twice: once to get the key list and once to iterate. Stored the result in a local variable to avoid the redundant dict reconstruction. 9. Lazy-import Upload and remove dead imports in manager.py. Moved `from webkitpy.results.upload import Upload` into the upload codepath where it's actually used. Removed completely unused `DeviceRequest` and `SimulatedDeviceManager` imports. * Tools/Scripts/webkitpy/api_tests/manager.py: (Manager): (Manager._is_exact_test_name): (Manager._try_collect_exact_tests): (Manager._collect_tests): (Manager._set_up_run): (Manager.run): * Tools/Scripts/webkitpy/api_tests/manager_unittest.py: (test_is_exact_test_name): * Tools/Scripts/webkitpy/port/base.py: (Port.check_api_test_build): (Port.commits_for_upload): * Tools/Scripts/webkitpy/port/embedded_device.py: (EmbeddedDevicePort.determine_full_port_name): * Tools/Scripts/webkitpy/port/factory.py: (configuration_options): (PortFactory): (PortFactory._load_port_class): (PortFactory.get): (PortFactory.get.in): * Tools/Scripts/webkitpy/port/ios_device.py: (IOSDevicePort): (IOSDevicePort.SDK): * Tools/Scripts/webkitpy/port/ios_simulator.py: (IOSSimulatorPort): (IOSSimulatorPort.SDK): * Tools/Scripts/webkitpy/port/port_testcase.py: (bind_mock_apple_additions.MockAppleAdditions): (bind_mock_apple_additions.MockAppleAdditions.get_sdk): * Tools/Scripts/webkitpy/port/visionos_device.py: (VisionOSDevicePort): (VisionOSDevicePort.SDK): * Tools/Scripts/webkitpy/port/visionos_simulator.py: (VisionOSSimulatorPort): (VisionOSSimulatorPort.SDK): * Tools/Scripts/webkitpy/port/watch_device.py: (WatchDevicePort): (WatchDevicePort.SDK): * Tools/Scripts/webkitpy/port/watch_simulator.py: (WatchSimulatorPort): (WatchSimulatorPort.SDK): Canonical link: https://commits.webkit.org/309636@main

webkit-commit-queue · 2026-03-20T21:43:08Z

Committed 309636@main (a80da14): https://commits.webkit.org/309636@main

Reviewed commits have been landed. Closing PR #60911 and removing active labels.

rr-codes requested review from JonWBedard and gsnedders as code owners March 19, 2026 04:03

rr-codes self-assigned this Mar 19, 2026

rr-codes added the WebKit Misc. For miscellaneous bugs in the WebKit framework (and not JavaScriptCore or WebCore). label Mar 19, 2026

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from b9a3d91 to e244c7f Compare March 19, 2026 04:16

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from e244c7f to 3b39240 Compare March 19, 2026 05:03

rr-codes requested review from aproskuryakov and emw-apple March 19, 2026 18:01

gsnedders reviewed Mar 19, 2026

View reviewed changes

Tools/Scripts/webkitpy/port/factory.py Show resolved Hide resolved

Tools/Scripts/webkitpy/port/factory.py Show resolved Hide resolved

Tools/Scripts/webkitpy/api_tests/manager.py Outdated Show resolved Hide resolved

Tools/Scripts/webkitpy/api_tests/manager.py Outdated Show resolved Hide resolved

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 3b39240 to 82a27e8 Compare March 19, 2026 20:58

rr-codes changed the title ~~[API Tests] Reduce run-api-tests startup overhead when running specific tests~~ [API Tests] Reduce run-api-tests startup overhead when running specific tests https://bugs.webkit.org/show_bug.cgi?id=310247 rdar://172887928 Mar 19, 2026

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 82a27e8 to 39ffd79 Compare March 19, 2026 21:42

rr-codes changed the title ~~[API Tests] Reduce run-api-tests startup overhead when running specific tests https://bugs.webkit.org/show_bug.cgi?id=310247 rdar://172887928~~ [API Tests] Reduce run-api-tests startup overhead when running specific tests Mar 19, 2026

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 39ffd79 to 21e9eff Compare March 19, 2026 21:59

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 21e9eff to 5a1eba7 Compare March 19, 2026 23:22

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 5a1eba7 to eac26fd Compare March 20, 2026 04:01

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from eac26fd to 9120ab2 Compare March 20, 2026 06:33

webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 20, 2026

rr-codes removed the merging-blocked Applied to prevent a change from being merged label Mar 20, 2026

JonWBedard approved these changes Mar 20, 2026

View reviewed changes

rr-codes added the merge-queue Applied to send a pull request to merge-queue label Mar 20, 2026

webkit-commit-queue added merging-blocked Applied to prevent a change from being merged and removed merge-queue Applied to send a pull request to merge-queue labels Mar 20, 2026

rr-codes removed the merging-blocked Applied to prevent a change from being merged label Mar 20, 2026

rr-codes force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 9120ab2 to 6e1cfbb Compare March 20, 2026 20:49

rr-codes added merge-queue Applied to send a pull request to merge-queue unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing and removed merge-queue Applied to send a pull request to merge-queue labels Mar 20, 2026

webkit-commit-queue force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 6e1cfbb to 6097dd8 Compare March 20, 2026 21:41

webkit-commit-queue force-pushed the eng/API-Tests-Reduce-run-api-tests-startup-overhead-when-running-specific-tests branch from 6097dd8 to a80da14 Compare March 20, 2026 21:43

webkit-commit-queue merged commit a80da14 into WebKit:main Mar 20, 2026

webkit-commit-queue removed the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 20, 2026

Conversation

rr-codes commented Mar 19, 2026 • edited by webkit-early-warning-system Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

a80da14

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gsnedders left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-commit-queue commented Mar 20, 2026

Uh oh!

webkit-early-warning-system commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-commit-queue commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

rr-codes commented Mar 19, 2026 •

edited by webkit-early-warning-system

Loading

`a80da14`

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 19, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 20, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 20, 2026 •

edited

Loading

webkit-early-warning-system commented Mar 20, 2026 •

edited

Loading