Skip to content

tests/test_wpt.py relies on Chrome Dev being installed? #53300

Closed
@gsnedders

Description

@gsnedders

https://github.com/web-platform-tests/wpt/actions/runs/15744999461/job/44379264561 failed:

2025-06-18T23:14:26.6543860Z + tox -f py313
2025-06-18T23:14:27.0085990Z py313: install_deps> python -I -m pip install -r /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/../requirements_pytest.txt -r /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/requirements.txt -r /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/../wptrunner/requirements.txt -r /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/../wptrunner/requirements_chromium.txt -r /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/../wptrunner/requirements_firefox.txt
2025-06-18T23:14:30.9114280Z py313: freeze> python -m pip freeze --all
2025-06-18T23:14:31.0553050Z py313: aioquic==1.2.0,attrs==25.3.0,blessed==1.21.0,certifi==2025.6.15,cffi==1.17.1,charset-normalizer==3.4.2,coverage==7.9.1,cryptography==45.0.4,distro==1.9.0,html5lib==1.1,hypothesis==6.112.0,idna==3.10,iniconfig==2.1.0,marionette-driver==3.4.0,mozcrash==2.2.1,mozdebug==0.3.1,mozdevice==4.2.0,mozfile==3.0.0,mozinfo==1.2.3,mozInstall==2.1.0,mozleak==0.2,mozlog==8.0.0,mozprocess==1.3.1,mozprofile==3.0.0,mozrunner==8.3.2,mozterm==1.0.0,mozversion==2.4.0,packaging==25.0,pillow==11.1.0,pip==25.1.1,pluggy==1.6.0,psutil==5.9.8,pyasn1==0.6.1,pyasn1_modules==0.4.2,pycparser==2.22,pylsqpack==0.3.22,pyOpenSSL==25.1.0,pytest==8.3.2,pytest-cov==5.0.0,redo==3.0.0,requests==2.32.3,service-identity==24.2.0,six==1.16.0,sortedcontainers==2.4.0,types-psutil==6.1.0.20241221,types-requests==2.32.0.20241016,types-six==1.17.0.20241205,urllib3==2.2.2,wcwidth==0.2.13,webencodings==0.5.1
2025-06-18T23:14:31.0556000Z py313: commands[0]> pytest -c/Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt/../pytest.ini --rootdir=/Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt
2025-06-18T23:14:32.3108180Z ============================= test session starts ==============================
2025-06-18T23:14:32.3108560Z platform darwin -- Python 3.13.4, pytest-8.3.2, pluggy-1.6.0
2025-06-18T23:14:32.3108790Z cachedir: .tox/py313/.pytest_cache
2025-06-18T23:14:32.3109010Z rootdir: /Users/buildbot/actions-runner/_work/wpt/wpt/tools/wpt
2025-06-18T23:14:32.3109330Z configfile: ../pytest.ini
2025-06-18T23:14:32.3109510Z plugins: cov-5.0.0, hypothesis-6.112.0
2025-06-18T23:14:32.3109680Z collected 84 items
2025-06-18T23:14:32.3109780Z 
2025-06-18T23:14:32.3444910Z tests/test_browser.py .............................                      [ 34%]
2025-06-18T23:14:52.0423750Z tests/test_install.py ....                                               [ 39%]
2025-06-18T23:14:52.0434140Z tests/test_markdown.py ...                                               [ 42%]
2025-06-18T23:14:52.0447970Z tests/test_revlist.py ...                                                [ 46%]
2025-06-18T23:14:55.9783720Z tests/test_run.py ..........s............                                [ 73%]
2025-06-18T23:14:56.2095680Z tests/test_testfiles.py ....                                             [ 78%]
2025-06-18T23:15:07.3845560Z tests/test_update_expectations.py .                                      [ 79%]
2025-06-18T23:15:50.9443770Z tests/test_wpt.py ...F..FFF........                                      [100%]
2025-06-18T23:15:50.9444010Z 
2025-06-18T23:15:50.9446350Z =================================== FAILURES ===================================
2025-06-18T23:15:50.9446870Z _______________________________ test_list_tests ________________________________
2025-06-18T23:15:50.9447050Z 
2025-06-18T23:15:50.9447250Z manifest_dir = '/var/folders/m7/v707hq2908b8dq6pnmt9gr280000gn/T/tmpmiz1tpsb'
2025-06-18T23:15:50.9447510Z 
2025-06-18T23:15:50.9447590Z     @pytest.mark.slow
2025-06-18T23:15:50.9447790Z     @pytest.mark.skipif(sys.platform == "win32",
2025-06-18T23:15:50.9448110Z                         reason="https://github.com/web-platform-tests/wpt/issues/28745")
2025-06-18T23:15:50.9448410Z     def test_list_tests(manifest_dir):
2025-06-18T23:15:50.9448700Z         """The `--list-tests` option should not produce an error under normal
2025-06-18T23:15:50.9449150Z         conditions."""
2025-06-18T23:15:50.9449380Z     
2025-06-18T23:15:50.9449550Z         with pytest.raises(SystemExit) as excinfo:
2025-06-18T23:15:50.9449870Z >           wpt.main(argv=["run", "--metadata", manifest_dir, "--list-tests",
2025-06-18T23:15:50.9450180Z                            "--channel", "dev", "--yes",
2025-06-18T23:15:50.9450860Z                            # WebTransport server is not needed (web-platform-tests/wpt#41675).
2025-06-18T23:15:50.9451160Z                            "--no-enable-webtransport-h3",
2025-06-18T23:15:50.9451540Z                            # Taskcluster machines do not have GPUs, so use software rendering via --enable-swiftshader.
2025-06-18T23:15:50.9451940Z                            "--enable-swiftshader",
2025-06-18T23:15:50.9452180Z                            "chrome", "/dom/nodes/Element-tagName.html"])
2025-06-18T23:15:50.9452350Z 
2025-06-18T23:15:50.9452410Z tests/test_wpt.py:126: 
2025-06-18T23:15:50.9452620Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9452830Z wpt.py:233: in main
2025-06-18T23:15:50.9453010Z     rv = script(*args, **kwargs)
2025-06-18T23:15:50.9453190Z run.py:1010: in run
2025-06-18T23:15:50.9453420Z     setup_cls, wptrunner_kwargs = setup_wptrunner(venv, **kwargs)
2025-06-18T23:15:50.9453700Z run.py:982: in setup_wptrunner
2025-06-18T23:15:50.9453880Z     setup_cls.setup(kwargs)
2025-06-18T23:15:50.9454070Z run.py:246: in setup
2025-06-18T23:15:50.9454250Z     self.setup_kwargs(kwargs)
2025-06-18T23:15:50.9454430Z run.py:491: in setup_kwargs
2025-06-18T23:15:50.9454630Z     webdriver_binary = self.browser.install_webdriver(
2025-06-18T23:15:50.9454890Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9455040Z 
2025-06-18T23:15:50.9455150Z self = <tools.wpt.browser.Chrome object at 0x10664a8a0>
2025-06-18T23:15:50.9455440Z dest = '/Users/buildbot/actions-runner/_work/wpt/wpt/_venv3/bin'
2025-06-18T23:15:50.9455690Z channel = 'dev'
2025-06-18T23:15:50.9455970Z browser_binary = '/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev'
2025-06-18T23:15:50.9456220Z 
2025-06-18T23:15:50.9456390Z     def install_webdriver(self, dest=None, channel=None, browser_binary=None):
2025-06-18T23:15:50.9456660Z         if dest is None:
2025-06-18T23:15:50.9456830Z             dest = os.pwd
2025-06-18T23:15:50.9456990Z     
2025-06-18T23:15:50.9457130Z         # Detect the browser version.
2025-06-18T23:15:50.9457370Z         # The ChromeDriver that is installed will match this version.
2025-06-18T23:15:50.9457610Z         if browser_binary is None:
2025-06-18T23:15:50.9457870Z             # If a browser binary path was not given, detect a valid path.
2025-06-18T23:15:50.9458130Z             browser_binary = self.find_binary(channel=channel)
2025-06-18T23:15:50.9458410Z             # We need a browser to version match, so if a browser binary path
2025-06-18T23:15:50.9458680Z             # was not given and cannot be detected, raise an error.
2025-06-18T23:15:50.9458910Z             if browser_binary is None:
2025-06-18T23:15:50.9459140Z                 raise FileNotFoundError("No browser binary detected. "
2025-06-18T23:15:50.9459680Z                                         "Cannot install ChromeDriver without a browser version.")
2025-06-18T23:15:50.9459970Z     
2025-06-18T23:15:50.9460130Z         version = self.version(browser_binary)
2025-06-18T23:15:50.9460320Z         if version is None:
2025-06-18T23:15:50.9460510Z             # Check if the user has given a Chromium binary.
2025-06-18T23:15:50.9460740Z             chromium = Chromium(self.logger)
2025-06-18T23:15:50.9460970Z             if chromium.version(browser_binary):
2025-06-18T23:15:50.9461270Z                 raise ValueError("Provided binary is a Chromium binary and should be run using "
2025-06-18T23:15:50.9461570Z                                  "\"./wpt run chromium\" or similar.")
2025-06-18T23:15:50.9461920Z >           raise ValueError(f"Unable to detect browser version from binary at {browser_binary}. "
2025-06-18T23:15:50.9462270Z                              " Cannot install ChromeDriver without a valid version to match.")
2025-06-18T23:15:50.9462890Z E           ValueError: Unable to detect browser version from binary at /Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev.  Cannot install ChromeDriver without a valid version to match.
2025-06-18T23:15:50.9463510Z 
2025-06-18T23:15:50.9463570Z browser.py:1378: ValueError
2025-06-18T23:15:50.9463800Z _____________________________ test_run_zero_tests ______________________________
2025-06-18T23:15:50.9463970Z 
2025-06-18T23:15:50.9464060Z     @pytest.mark.slow
2025-06-18T23:15:50.9464240Z     @pytest.mark.remote_network
2025-06-18T23:15:50.9464470Z     @pytest.mark.skipif(sys.platform == "win32",
2025-06-18T23:15:50.9464740Z                         reason="https://github.com/web-platform-tests/wpt/issues/28745")
2025-06-18T23:15:50.9465030Z     def test_run_zero_tests():
2025-06-18T23:15:50.9465270Z         """A test execution describing zero tests should be reported as an error
2025-06-18T23:15:50.9465630Z         even in the presence of the `--no-fail-on-unexpected` option."""
2025-06-18T23:15:50.9465880Z         if is_port_8000_in_use():
2025-06-18T23:15:50.9466060Z             pytest.skip("port 8000 already in use")
2025-06-18T23:15:50.9466260Z     
2025-06-18T23:15:50.9466400Z         with pytest.raises(SystemExit) as excinfo:
2025-06-18T23:15:50.9466690Z >           wpt.main(argv=["run", "--yes", "--no-pause", "--channel", "dev",
2025-06-18T23:15:50.9467000Z                            # WebTransport server is not needed (web-platform-tests/wpt#41675).
2025-06-18T23:15:50.9467310Z                            "--no-enable-webtransport-h3",
2025-06-18T23:15:50.9467660Z                            # Taskcluster machines do not have GPUs, so use software rendering via --enable-swiftshader.
2025-06-18T23:15:50.9467970Z                            "--enable-swiftshader",
2025-06-18T23:15:50.9468200Z                            "chrome", "/non-existent-dir/non-existent-file.html"])
2025-06-18T23:15:50.9468370Z 
2025-06-18T23:15:50.9468420Z tests/test_wpt.py:208: 
2025-06-18T23:15:50.9468610Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9468820Z wpt.py:233: in main
2025-06-18T23:15:50.9468960Z     rv = script(*args, **kwargs)
2025-06-18T23:15:50.9469160Z run.py:1010: in run
2025-06-18T23:15:50.9469340Z     setup_cls, wptrunner_kwargs = setup_wptrunner(venv, **kwargs)
2025-06-18T23:15:50.9469610Z run.py:982: in setup_wptrunner
2025-06-18T23:15:50.9469810Z     setup_cls.setup(kwargs)
2025-06-18T23:15:50.9470010Z run.py:246: in setup
2025-06-18T23:15:50.9470150Z     self.setup_kwargs(kwargs)
2025-06-18T23:15:50.9470310Z run.py:491: in setup_kwargs
2025-06-18T23:15:50.9470500Z     webdriver_binary = self.browser.install_webdriver(
2025-06-18T23:15:50.9470740Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9470880Z 
2025-06-18T23:15:50.9470990Z self = <tools.wpt.browser.Chrome object at 0x1323157c0>
2025-06-18T23:15:50.9471250Z dest = '/Users/buildbot/actions-runner/_work/wpt/wpt/_venv3/bin'
2025-06-18T23:15:50.9471620Z channel = 'dev'
2025-06-18T23:15:50.9471870Z browser_binary = '/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev'
2025-06-18T23:15:50.9472220Z 
2025-06-18T23:15:50.9472350Z     def install_webdriver(self, dest=None, channel=None, browser_binary=None):
2025-06-18T23:15:50.9472620Z         if dest is None:
2025-06-18T23:15:50.9472770Z             dest = os.pwd
2025-06-18T23:15:50.9472920Z     
2025-06-18T23:15:50.9473110Z         # Detect the browser version.
2025-06-18T23:15:50.9473340Z         # The ChromeDriver that is installed will match this version.
2025-06-18T23:15:50.9473570Z         if browser_binary is None:
2025-06-18T23:15:50.9473790Z             # If a browser binary path was not given, detect a valid path.
2025-06-18T23:15:50.9474050Z             browser_binary = self.find_binary(channel=channel)
2025-06-18T23:15:50.9474350Z             # We need a browser to version match, so if a browser binary path
2025-06-18T23:15:50.9474670Z             # was not given and cannot be detected, raise an error.
2025-06-18T23:15:50.9474890Z             if browser_binary is None:
2025-06-18T23:15:50.9475280Z                 raise FileNotFoundError("No browser binary detected. "
2025-06-18T23:15:50.9475570Z                                         "Cannot install ChromeDriver without a browser version.")
2025-06-18T23:15:50.9475780Z     
2025-06-18T23:15:50.9501330Z         version = self.version(browser_binary)
2025-06-18T23:15:50.9501520Z         if version is None:
2025-06-18T23:15:50.9501690Z             # Check if the user has given a Chromium binary.
2025-06-18T23:15:50.9501900Z             chromium = Chromium(self.logger)
2025-06-18T23:15:50.9502090Z             if chromium.version(browser_binary):
2025-06-18T23:15:50.9502370Z                 raise ValueError("Provided binary is a Chromium binary and should be run using "
2025-06-18T23:15:50.9502660Z                                  "\"./wpt run chromium\" or similar.")
2025-06-18T23:15:50.9503000Z >           raise ValueError(f"Unable to detect browser version from binary at {browser_binary}. "
2025-06-18T23:15:50.9503340Z                              " Cannot install ChromeDriver without a valid version to match.")
2025-06-18T23:15:50.9503930Z E           ValueError: Unable to detect browser version from binary at /Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev.  Cannot install ChromeDriver without a valid version to match.
2025-06-18T23:15:50.9504400Z 
2025-06-18T23:15:50.9504450Z browser.py:1378: ValueError
2025-06-18T23:15:50.9504650Z ____________________________ test_run_failing_test _____________________________
2025-06-18T23:15:50.9504820Z 
2025-06-18T23:15:50.9504870Z     @pytest.mark.slow
2025-06-18T23:15:50.9505030Z     @pytest.mark.remote_network
2025-06-18T23:15:50.9505210Z     @pytest.mark.skipif(sys.platform == "win32",
2025-06-18T23:15:50.9505510Z                         reason="https://github.com/web-platform-tests/wpt/issues/28745")
2025-06-18T23:15:50.9505760Z     def test_run_failing_test():
2025-06-18T23:15:50.9506040Z         """Failing tests should be reported with a non-zero exit status unless the
2025-06-18T23:15:50.9506370Z         `--no-fail-on-unexpected` option has been specified."""
2025-06-18T23:15:50.9506580Z         if is_port_8000_in_use():
2025-06-18T23:15:50.9506760Z             pytest.skip("port 8000 already in use")
2025-06-18T23:15:50.9507040Z         failing_test = "/infrastructure/expected-fail/failing-test.html"
2025-06-18T23:15:50.9507260Z     
2025-06-18T23:15:50.9507400Z         assert os.path.isfile("../../%s" % failing_test)
2025-06-18T23:15:50.9507570Z     
2025-06-18T23:15:50.9507710Z         with pytest.raises(SystemExit) as excinfo:
2025-06-18T23:15:50.9507940Z >           wpt.main(argv=["run", "--yes", "--no-pause", "--channel", "dev",
2025-06-18T23:15:50.9508230Z                            # WebTransport server is not needed (web-platform-tests/wpt#41675).
2025-06-18T23:15:50.9508520Z                            "--no-enable-webtransport-h3",
2025-06-18T23:15:50.9509060Z                            # Taskcluster machines do not have GPUs, so use software rendering via --enable-swiftshader.
2025-06-18T23:15:50.9509400Z                            "--enable-swiftshader",
2025-06-18T23:15:50.9509630Z                            "chrome", failing_test])
2025-06-18T23:15:50.9509780Z 
2025-06-18T23:15:50.9509830Z tests/test_wpt.py:241: 
2025-06-18T23:15:50.9510050Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9510250Z wpt.py:233: in main
2025-06-18T23:15:50.9510520Z     rv = script(*args, **kwargs)
2025-06-18T23:15:50.9510680Z run.py:1010: in run
2025-06-18T23:15:50.9510900Z     setup_cls, wptrunner_kwargs = setup_wptrunner(venv, **kwargs)
2025-06-18T23:15:50.9511120Z run.py:982: in setup_wptrunner
2025-06-18T23:15:50.9511290Z     setup_cls.setup(kwargs)
2025-06-18T23:15:50.9511430Z run.py:246: in setup
2025-06-18T23:15:50.9511580Z     self.setup_kwargs(kwargs)
2025-06-18T23:15:50.9511770Z run.py:491: in setup_kwargs
2025-06-18T23:15:50.9511950Z     webdriver_binary = self.browser.install_webdriver(
2025-06-18T23:15:50.9512210Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9512470Z 
2025-06-18T23:15:50.9512570Z self = <tools.wpt.browser.Chrome object at 0x122534ad0>
2025-06-18T23:15:50.9512850Z dest = '/Users/buildbot/actions-runner/_work/wpt/wpt/_venv3/bin'
2025-06-18T23:15:50.9513070Z channel = 'dev'
2025-06-18T23:15:50.9513300Z browser_binary = '/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev'
2025-06-18T23:15:50.9513530Z 
2025-06-18T23:15:50.9513680Z     def install_webdriver(self, dest=None, channel=None, browser_binary=None):
2025-06-18T23:15:50.9513920Z         if dest is None:
2025-06-18T23:15:50.9514080Z             dest = os.pwd
2025-06-18T23:15:50.9514220Z     
2025-06-18T23:15:50.9514360Z         # Detect the browser version.
2025-06-18T23:15:50.9514570Z         # The ChromeDriver that is installed will match this version.
2025-06-18T23:15:50.9514780Z         if browser_binary is None:
2025-06-18T23:15:50.9514990Z             # If a browser binary path was not given, detect a valid path.
2025-06-18T23:15:50.9515230Z             browser_binary = self.find_binary(channel=channel)
2025-06-18T23:15:50.9515490Z             # We need a browser to version match, so if a browser binary path
2025-06-18T23:15:50.9515730Z             # was not given and cannot be detected, raise an error.
2025-06-18T23:15:50.9515950Z             if browser_binary is None:
2025-06-18T23:15:50.9516200Z                 raise FileNotFoundError("No browser binary detected. "
2025-06-18T23:15:50.9516450Z                                         "Cannot install ChromeDriver without a browser version.")
2025-06-18T23:15:50.9516720Z     
2025-06-18T23:15:50.9516900Z         version = self.version(browser_binary)
2025-06-18T23:15:50.9517080Z         if version is None:
2025-06-18T23:15:50.9517250Z             # Check if the user has given a Chromium binary.
2025-06-18T23:15:50.9517480Z             chromium = Chromium(self.logger)
2025-06-18T23:15:50.9517720Z             if chromium.version(browser_binary):
2025-06-18T23:15:50.9518010Z                 raise ValueError("Provided binary is a Chromium binary and should be run using "
2025-06-18T23:15:50.9518290Z                                  "\"./wpt run chromium\" or similar.")
2025-06-18T23:15:50.9518600Z >           raise ValueError(f"Unable to detect browser version from binary at {browser_binary}. "
2025-06-18T23:15:50.9518930Z                              " Cannot install ChromeDriver without a valid version to match.")
2025-06-18T23:15:50.9519550Z E           ValueError: Unable to detect browser version from binary at /Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev.  Cannot install ChromeDriver without a valid version to match.
2025-06-18T23:15:50.9519990Z 
2025-06-18T23:15:50.9520060Z browser.py:1378: ValueError
2025-06-18T23:15:50.9520400Z ___________________________ test_run_verify_unstable ___________________________
2025-06-18T23:15:50.9520610Z 
2025-06-18T23:15:50.9520710Z temp_test = <function temp_test.<locals>.make_test at 0x122df1580>
2025-06-18T23:15:50.9520890Z 
2025-06-18T23:15:50.9520960Z     @pytest.mark.slow
2025-06-18T23:15:50.9521120Z     @pytest.mark.remote_network
2025-06-18T23:15:50.9521310Z     @pytest.mark.skipif(sys.platform == "win32",
2025-06-18T23:15:50.9521610Z                         reason="https://github.com/web-platform-tests/wpt/issues/28745")
2025-06-18T23:15:50.9521870Z     def test_run_verify_unstable(temp_test):
2025-06-18T23:15:50.9522130Z         """Unstable tests should be reported with a non-zero exit status. Stable
2025-06-18T23:15:50.9522400Z         tests should be reported with a zero exit status."""
2025-06-18T23:15:50.9522610Z         if is_port_8000_in_use():
2025-06-18T23:15:50.9522790Z             pytest.skip("port 8000 already in use")
2025-06-18T23:15:50.9522980Z         unstable_test = temp_test("""
2025-06-18T23:15:50.9523190Z             test(function() {
2025-06-18T23:15:50.9523390Z                 if (localStorage.getItem('wpt-unstable-test-flag')) {
2025-06-18T23:15:50.9523780Z                   throw new Error();
2025-06-18T23:15:50.9523930Z                 }
2025-06-18T23:15:50.9524090Z     
2025-06-18T23:15:50.9524260Z                 localStorage.setItem('wpt-unstable-test-flag', 'x');
2025-06-18T23:15:50.9524460Z             }, 'my test');
2025-06-18T23:15:50.9524610Z         """)
2025-06-18T23:15:50.9524740Z     
2025-06-18T23:15:50.9524890Z         with pytest.raises(SystemExit) as excinfo:
2025-06-18T23:15:50.9525130Z >           wpt.main(argv=["run", "--yes", "--verify", "--channel", "dev",
2025-06-18T23:15:50.9525410Z                            # WebTransport server is not needed (web-platform-tests/wpt#41675).
2025-06-18T23:15:50.9525700Z                            "--no-enable-webtransport-h3",
2025-06-18T23:15:50.9526010Z                            # Taskcluster machines do not have GPUs, so use software rendering via --enable-swiftshader.
2025-06-18T23:15:50.9526300Z                            "--enable-swiftshader",
2025-06-18T23:15:50.9526500Z                            "chrome", unstable_test])
2025-06-18T23:15:50.9526640Z 
2025-06-18T23:15:50.9526700Z tests/test_wpt.py:280: 
2025-06-18T23:15:50.9526890Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9527110Z wpt.py:233: in main
2025-06-18T23:15:50.9527250Z     rv = script(*args, **kwargs)
2025-06-18T23:15:50.9527410Z run.py:1010: in run
2025-06-18T23:15:50.9527630Z     setup_cls, wptrunner_kwargs = setup_wptrunner(venv, **kwargs)
2025-06-18T23:15:50.9527850Z run.py:982: in setup_wptrunner
2025-06-18T23:15:50.9528050Z     setup_cls.setup(kwargs)
2025-06-18T23:15:50.9528220Z run.py:246: in setup
2025-06-18T23:15:50.9528370Z     self.setup_kwargs(kwargs)
2025-06-18T23:15:50.9528510Z run.py:491: in setup_kwargs
2025-06-18T23:15:50.9528700Z     webdriver_binary = self.browser.install_webdriver(
2025-06-18T23:15:50.9528980Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2025-06-18T23:15:50.9529120Z 
2025-06-18T23:15:50.9529230Z self = <tools.wpt.browser.Chrome object at 0x122552430>
2025-06-18T23:15:50.9529490Z dest = '/Users/buildbot/actions-runner/_work/wpt/wpt/_venv3/bin'
2025-06-18T23:15:50.9529720Z channel = 'dev'
2025-06-18T23:15:50.9529970Z browser_binary = '/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev'
2025-06-18T23:15:50.9530320Z 
2025-06-18T23:15:50.9530470Z     def install_webdriver(self, dest=None, channel=None, browser_binary=None):
2025-06-18T23:15:50.9530710Z         if dest is None:
2025-06-18T23:15:50.9530870Z             dest = os.pwd
2025-06-18T23:15:50.9530990Z     
2025-06-18T23:15:50.9531190Z         # Detect the browser version.
2025-06-18T23:15:50.9531450Z         # The ChromeDriver that is installed will match this version.
2025-06-18T23:15:50.9531660Z         if browser_binary is None:
2025-06-18T23:15:50.9532060Z             # If a browser binary path was not given, detect a valid path.
2025-06-18T23:15:50.9532320Z             browser_binary = self.find_binary(channel=channel)
2025-06-18T23:15:50.9532600Z             # We need a browser to version match, so if a browser binary path
2025-06-18T23:15:50.9532850Z             # was not given and cannot be detected, raise an error.
2025-06-18T23:15:50.9533050Z             if browser_binary is None:
2025-06-18T23:15:50.9533260Z                 raise FileNotFoundError("No browser binary detected. "
2025-06-18T23:15:50.9533540Z                                         "Cannot install ChromeDriver without a browser version.")
2025-06-18T23:15:50.9533760Z     
2025-06-18T23:15:50.9533890Z         version = self.version(browser_binary)
2025-06-18T23:15:50.9534070Z         if version is None:
2025-06-18T23:15:50.9534240Z             # Check if the user has given a Chromium binary.
2025-06-18T23:15:50.9534450Z             chromium = Chromium(self.logger)
2025-06-18T23:15:50.9534640Z             if chromium.version(browser_binary):
2025-06-18T23:15:50.9534940Z                 raise ValueError("Provided binary is a Chromium binary and should be run using "
2025-06-18T23:15:50.9535370Z                                  "\"./wpt run chromium\" or similar.")
2025-06-18T23:15:50.9535670Z >           raise ValueError(f"Unable to detect browser version from binary at {browser_binary}. "
2025-06-18T23:15:50.9536030Z                              " Cannot install ChromeDriver without a valid version to match.")
2025-06-18T23:15:50.9536600Z E           ValueError: Unable to detect browser version from binary at /Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev.  Cannot install ChromeDriver without a valid version to match.
2025-06-18T23:15:50.9537070Z 
2025-06-18T23:15:50.9537120Z browser.py:1378: ValueError
2025-06-18T23:15:50.9537320Z =========================== short test summary info ============================
2025-06-18T23:15:50.9537720Z FAILED tests/test_wpt.py::test_list_tests - ValueError: Unable to detect brow...
2025-06-18T23:15:50.9538060Z FAILED tests/test_wpt.py::test_run_zero_tests - ValueError: Unable to detect ...
2025-06-18T23:15:50.9538410Z FAILED tests/test_wpt.py::test_run_failing_test - ValueError: Unable to detec...
2025-06-18T23:15:50.9538740Z FAILED tests/test_wpt.py::test_run_verify_unstable - ValueError: Unable to de...
2025-06-18T23:15:50.9539030Z ============== 4 failed, 79 passed, 1 skipped in 79.13s (0:01:19) ==============

There doesn't seem to be anything that would actually install Chrome Dev; these tests just rely on it being installed? I wonder if this was where I started with #53299 — and also whether that code path is just broken on macOS?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions