Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🚀 Feature]: Error when Selenium Manager used on unsupported architecture #11599

Closed
titusfortner opened this issue Jan 30, 2023 · 34 comments
Closed

Comments

@titusfortner
Copy link
Member

titusfortner commented Jan 30, 2023

Feature and motivation

Right now, Selenium Manager is called when a driver isn't specified or found on PATH. We only support 3 architectures, though. We should add code to explicitly check the user's architecture in preparation for additional additional architectures and throw an error until the architecture is actually supported.

This isn't as much of a problem while the Manager is in a "fall back" mode, but it can cause unexpected/confusing errors especially once we stop hiding them.

@titusfortner
Copy link
Member Author

Turns out that Mac ARM architecture works just fine with AMD Selenium Manager, so that one isn't an issue.

So, I'm going off of this list here: #11357

There are drivers available in various places for each of these architectures, but our current selenium manager binaries won't even run on them:
linux-32
win32
linux-arm64
win-arm64
linux-armv7

If we checked in a 32 bit version of Selenium Manager for Windows & Linux, it should also work for 64 bit OS, right?

@titusfortner
Copy link
Member Author

Here are some rough numbers for geckodriver downloads:

Architecture Percent Currently Supported
linux64 51%
win64 24%
macos 9%
macos-aarch64 5%
win32 5%
win-aarch64 2%
linux-aarch64 2%
linux32 1%

So presumably we are currently supporting "only" ~90% of usage

@bonigarcia
Copy link
Member

To clarify these figures a bit, the Selenium Manager Rust code already supports every platform in geckodriver (including win32, linux32, etc.). But it is true that the current compilation is only for win64, linux64, and mac64. So win32 and linux32 cannot be downloaded with Selenium Manager since the binary cannot be executed. But I have doubts ARM64 in Windows and Linux. Maybe the x64 binary can be executed in Windows and Linux (like in macOS). In that case, the win-aarch64 and linux-aarch64 are also supported. Unfortunately, I don't have an ARM64 machine for Windows or Linux to check it.

@titusfortner
Copy link
Member Author

I'm almost certain arm64 will need to be supported separately for Linux/Windows.

So specifically for next release we need to add to the bindings checks with errors for the 4 above that can't be obtained automatically via Selenium right now.

I'm *hoping we might be able to build & bundle 32 bit versions of Linux/Windows and that they will still work for 64 bit Linux/Windows. That would get us to > 95% at least, which is good enough until we figure out cross compilation.

@diemol
Copy link
Member

diemol commented Mar 29, 2023

We had an informal chat a couple of days ago at SeleniumConf, where we realized that Windows 32 is going away and new versions are not coming in that flavor anymore.

I tried to get a more "official" link, but this has the gist of it
https://www.bleepingcomputer.com/news/microsoft/rip-microsoft-to-drop-support-for-windows-10-on-32-bit-systems/

I would not want to support something that is slowly going away. We can add a FAQ in the docs showing what to do if you have a Windows 32 machine. What do you think?

@titusfortner
Copy link
Member Author

I don't think this is an issue any longer because our current binary supports win32 & win64, but...

  1. All drivers support a 32 bit windows version, and so long as they offer the version, Selenium manager should pick it as appropriate
  2. Firefox sees ~5% of users requesting the 32 bit version, so it isn't zero.

At this point we only need to raise an error for "not x86 and not Apple" and Linux 32 bit

@diemol
Copy link
Member

diemol commented Apr 4, 2023

What is missing to close this issue?

@titusfortner
Copy link
Member Author

We want the bindings to check for unsupported architectures and throw an error saying the architecture isn't supported.

@diemol
Copy link
Member

diemol commented Apr 5, 2023

Shouldn't the process invoked by the bindings return something like that? We are already printing the error.

@titusfortner
Copy link
Member Author

We shouldn't try to send a command to an executable if we know it is the wrong executable, and we can be explicit about how we don't support that architecture so the user knows what is wrong.

@diemol
Copy link
Member

diemol commented Apr 5, 2023

That is a good point. Right now with the amount of people we have around to work on this I would just delegate it to the OS and then print the message. But let's leave this open and see if someone can help. Low priority in my opinion.

@diemol diemol added this to the 4.10 milestone Apr 17, 2023
@titusfortner
Copy link
Member Author

Alex says that the packaged binary works on ARM64 Windows as expected, which I think means the 3 binaries we have might be sufficient for everything. And at this point, I think if it doesn't work, we should wait to get feedback from users that it isn't working and what the error is. I was concerned the error a user got wouldn't be good enough, but if we can't even generate an error, this isn't necessary as a proactive measure.

@ychaouche
Copy link

I am a firefox user on a 32 bit linux system.
selenium used to work a few years ago.
I tried it today but couldn't get it to work because of this error.
Should I just use an older version of selenium?

python 3.11
selenium 4.9.1

@titusfortner
Copy link
Member Author

@ychaouche what was the error?
If it is Selenium Manager related, can you turn on logging so we can see what might be happening? https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

@ychaouche
Copy link

ychaouche commented Jun 4, 2023

Hello @titusfortner,
thanks for reaching back,
and sorry for delay.

Original code and traceback follows:


19:58:14 ~/CODE/TEST/PYTHON -2- $ cat selenium3.py
import selenium.webdriver
browser_options = selenium.webdriver.FirefoxOptions()
browser_options.add_argument("-headless")
browser = selenium.webdriver.Firefox(options=browser_options)
print(browser)

19:58:18 ~/CODE/TEST/PYTHON -2- $  
16:10:16 ~/CODE/TEST/PYTHON -2- $ python3 selenium3.py
Traceback (most recent call last):
  File "/home/home/chaouche/CODE/TEST/PYTHON/selenium3.py", line 4, in <module>
    browser = selenium.webdriver.Firefox(options=browser_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/firefox/webdriver.py", line 195, in __init__
    self.service.path = DriverFinder.get_path(self.service, options)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 40, in get_path
    path = shutil.which(service.path) or SeleniumManager().driver_location(options)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 91, in driver_location
    result = self.run(args)
             ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 106, in run
    completed_proc = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1917, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager'

selenium-manager is ELF 64

19:59:59 ~/CODE/TEST/PYTHON -2- $ file /usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager
/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped
20:00:02 ~/CODE/TEST/PYTHON -2- $   

I will read the given link and see how I can improve logging.

@ychaouche
Copy link

Ok here's output with debug info from selenium
after adding

import logging
logging.basiConfig(level=logging.DEBUG)

to my test code

20:44:19 ~/CODE/TEST/PYTHON -2- $ python3 selenium3.py
INFO:selenium.webdriver.common.selenium_manager:Applicable driver not found; attempting to install with Selenium Manager (Beta)
DEBUG:selenium.webdriver.common.selenium_manager:Executing process: /usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager --browser firefox --output json --debug
Traceback (most recent call last):
  File "/home/home/chaouche/CODE/TEST/PYTHON/selenium3.py", line 7, in <module>
    browser = selenium.webdriver.Firefox(options=browser_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/firefox/webdriver.py", line 195, in __init__
    self.service.path = DriverFinder.get_path(self.service, options)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 40, in get_path
    path = shutil.which(service.path) or SeleniumManager().driver_location(options)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 91, in driver_location
    result = self.run(args)
             ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 106, in run
    completed_proc = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1917, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager'
20:45:15 ~/CODE/TEST/PYTHON -2- $  

@ychaouche
Copy link

ychaouche commented Jun 4, 2023

Added a firefox_bin param,

ffbin="~/DOWNLOADS/APPLICATIONS/firefox/firefox"
[...]
browser = selenium.webdriver.Firefox(options=browser_options,firefox_binary=ffbin)

but it says it's deprecated,

20:49:06 ~/CODE/TEST/PYTHON -2- $ python3 selenium3.py
/home/home/chaouche/CODE/TEST/PYTHON/selenium3.py:9: DeprecationWarning: firefox_binary has been deprecated, please pass in a Service object
[...]

then same error message is displayed.
So now I need to read on that subject.

@ychaouche
Copy link

ychaouche commented Jun 4, 2023

I tried to use a service,
but it still can't find it.

21:20:04 ~/CODE/TEST/PYTHON -2- $ cat selenium3.py
import selenium.webdriver
from selenium.webdriver.firefox.service import Service as geckoService
import logging

logging.basicConfig(level=logging.DEBUG)


gecko_service=geckoService("~/DOWNLOADS/APPLICATIONS/firefox/geckodriver")
ffpath="~/DOWNLOADS/APPLICATIONS/firefox/firefox"
browser_options = selenium.webdriver.FirefoxOptions()
browser_options.add_argument("-headless")
browser = selenium.webdriver.Firefox(options=browser_options,service=gecko_service)
print(browser)


21:20:24 ~/CODE/TEST/PYTHON -2- $ python3 selenium3.py
INFO:selenium.webdriver.common.selenium_manager:Applicable driver not found; attempting to install with Selenium Manager (Beta)
DEBUG:selenium.webdriver.common.selenium_manager:Executing process: /usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager --browser firefox --output json --debug
Traceback (most recent call last):
  File "/home/home/chaouche/CODE/TEST/PYTHON/selenium3.py", line 12, in <module>
    browser = selenium.webdriver.Firefox(options=browser_options,service=gecko_service)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/firefox/webdriver.py", line 195, in __init__
    self.service.path = DriverFinder.get_path(self.service, options)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 40, in get_path
    path = shutil.which(service.path) or SeleniumManager().driver_location(options)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 91, in driver_location
    result = self.run(args)
             ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 106, in run
    completed_proc = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1917, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager'
21:20:30 ~/CODE/TEST/PYTHON -2- $   

@ychaouche
Copy link

ychaouche commented Jun 4, 2023

Oh...
I was using "~"
I'm so used to writing bash scripts.
I just converted the "~" to "/home/chaouche"
now it runs fine.

Sorry for being a little verbose,
and thanks for making me hope there was a solution to this problem!

Issue solved.

@titusfortner
Copy link
Member Author

For the record, we've deprecated a bunch of things for 4.0, and are removing them for 4.10.
Essentially:
Options class --> Browser Related
Service class --> Driver Related

So, setting Firefox Binary is:

    options = FirefoxOptions()
    options.binary="/path/to/firefox"
    driver = webdriver.Firefox(options=options)

@ychaouche
Copy link

ychaouche commented Jun 4, 2023

Thanks for the headsup.
I am using 4.9.1
how about setting the path to gecko?
would that be a DriverOption?

@titusfortner
Copy link
Member Author

setting path to the driver is a Service function. But, Selenium should be doing that for you automatically now.

@ychaouche
Copy link

I guess it will search in the PATH,
but gecko isn't there.
But it did find firefox!
So for now,
my working code is:

import selenium.webdriver
from selenium.webdriver.firefox.service import Service as geckoService

gecko_service=geckoService("/home/chaouche/DOWNLOADS/APPLICATIONS/firefox/geckodriver")
browser_options = selenium.webdriver.FirefoxOptions()
browser_options.add_argument("-P empty")
browser = selenium.webdriver.Firefox(options=browser_options,service=gecko_service)
print(browser)

@cml37
Copy link

cml37 commented Aug 5, 2023

I just upgraded to 4.11.0, which I think invokes Selenium Manager by default. I'm running on a Raspberry Pi (armv7l). What I am seeing is deployment of the selenium-manager linux binary and attempts to execute it, which naturally isn't going to work since I presume it is x64.

In at least one execution run, the executable got left on disk as well... guessing I did something that bypassed the shutdown hook 😅

But since it is there, it makes for an easy execution test:

pi@soundpi:/tmp/selenium-manager5240944676209986443430991726236463 $ ./selenium-manager
-bash: ./selenium-manager: cannot execute binary file: Exec format error

And naturally

pi@soundpi:/tmp $ uname -m
armv7l

Going to pass in a path to work around this, but just wanted to flag that invoking a ChromeDriver in Java land is resulting in attempt to execute a linux64 binary.

@titusfortner
Copy link
Member Author

@cml37 thanks for letting us know. Since I don't have a armv7l to test on, is there a condition that works to distinguish your architecture?
The only difference would be that we'd throw an error message saying it isn't supported and to pass in the location directly instead of relying on PATH.

@cml37
Copy link

cml37 commented Aug 5, 2023

Good question! Perhaps with Apache Commons, though I haven't tried it! https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/ArchUtils.html

@bonigarcia
Copy link
Member

I experimented with reproducing the same conditions that @cml37 commented. For that, I manually changed the binary contained in the Selenium 4.11.0 Java distribution (i.e., the file selenium-manager-4.11.0.jar). I changed the Windows binary to an incorrect one (e.g., macOS) to avoid executing it properly from Windows. Additionally, I put the correct chromedriver in PATH.

The problem is that the bindings always try to execute Selenium Manager, even when the driver is the PATH. After execution, Selenium Manager reports the driver to be used (on the PATH or download) on result.driver_path. In the case of an incorrect driver on PATH, it displays a warning if the driver is incompatible. But the problem here is that Selenium Manager is not even executed since the architecture is non-compliant (in my case, it is a forced error). And the bindings re-throw this error to the user.

org.openqa.selenium.remote.NoSuchDriverException: Unable to obtain: Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], extensions: []}}, error Failed to run command: [C:\Users\boni\AppData\Local\Temp\selenium-manager35659171830016161763349189133207\selenium-manager.exe, --browser, chrome, --output, json, --debug]
Build info: version: '4.11.0', revision: '040bc5406b'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.15'
Driver info: driver.version: ChromeDriver

...

Caused by: org.openqa.selenium.WebDriverException: Failed to run command: [C:\Users\boni\AppData\Local\Temp\selenium-manager35659171830016161763349189133207\selenium-manager.exe, --browser, chrome, --output, json, --debug]
Build info: version: '4.11.0', revision: '040bc5406b'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.15'
Driver info: driver.version: ChromeDriver
	at org.openqa.selenium.manager.SeleniumManager.runCommand(SeleniumManager.java:124)
	at org.openqa.selenium.manager.SeleniumManager.getDriverPath(SeleniumManager.java:273)
	at org.openqa.selenium.remote.service.DriverFinder.getPath(DriverFinder.java:22)
	... 80 more
Caused by: java.io.IOException: Cannot run program "C:\Users\boni\AppData\Local\Temp\selenium-manager35659171830016161763349189133207\selenium-manager.exe": CreateProcess error=216, This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher
...

I think the bindings (at least with Java, but perhaps it happens with the rest of the bindings) should do better and not error when the Selenium Manager binary cannot be executed. Otherwise, as it happens @cml37, Selenium cannot be executed in architectures like armv7l or any other (not even putting the driver on the PATH).

@cml37
Copy link

cml37 commented Aug 6, 2023

@bonigarcia as it ends up, you have to actually set the executable, see here: #11356 (comment)

@titusfortner
Copy link
Member Author

Yes, this issue was written with the idea that each of the bindings would throw the error rather than relying on Selenium manager binary.

But "not compatible with the operating system" seems like a decent error? Not sure what it is in other bindings.

We can reopen this if we want bindings to check for unsupported architectures and handle the error if we think that's better. But none of the other driver managers support these architectures, so seems reasonable that people would be managing them manually anyway.

@chadlwilson
Copy link

chadlwilson commented Sep 1, 2023

There are similar problems to @cml37's with the Ruby bindings on all aarch64 platforms because there are no platform specific gems published and https://rubygems.org/gems/selenium-webdriver/versions/4.11.0 includes only x64 binaries for each platform.

Error Message: Unable to obtain geckodriver using Selenium Manager; Unsuccessful command executed: ["/go/pipelines/smoke/vendor/bundle/ruby/3.2.0/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager", "--browser", "firefox", "--output", "json"]
/go/pipelines/smoke/vendor/bundle/ruby/3.2.0/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager: /go/pipelines/smoke/vendor/bundle/ruby/3.2.0/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager: cannot execute binary file
; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

@titusfortner
Copy link
Member Author

4.12 improves direct support for Mac aarch; the windows binary should work 32, 64 and aarch. As for Linux, Google doesn't release 32 bit drivers, and 32 bit Linux Firefox is 1% of geckodriver downloads.

Users can add the driver path to a service class if selenium does not provide direct support.

@nicolaslouge
Copy link

Thanks Titus for the comment above, it helped me to finally set everything up on my Raspberry Pi 4.

For anyone landing on this thread after searching for the error code, here's what the service class looks like if you have to use it in your Python script:

from selenium import webdriver

service = webdriver.ChromeService(executable_path='/usr/local/bin/geckodriver')
driver = webdriver.Firefox(service=service)

driver.get("https://duck.com/")

If you need to set up geckodriver and Selenium on a Raspberry Pi, I wrote the following blog - https://nicolaslouge.com/post/how-to-set-up-selenium-python-geckodriver-raspberry-pi-arm-2023/

@titusfortner
Copy link
Member Author

There's an alternate solution here: #12651 (comment)

Copy link

github-actions bot commented Dec 4, 2023

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Status: Done
Development

No branches or pull requests

7 participants