Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chrome not reachable with Selenium Python #133

Closed
robroc opened this issue Apr 16, 2018 · 15 comments

Comments

Projects
None yet
7 participants
@robroc
Copy link

commented Apr 16, 2018

I'm using serverless-chrome with Python in Lambda, following this repo's instructions.

It works great locally in a Docker container, but not when deployed to Lambda. This error comes up. It seems the webdriver starts up fine but the binary is unreachable when asked to parse the DOM with Selenium.

(Session info: headless chrome=65.0.3325.146)
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.9.85-38.58.amzn1.x86_64 x86_64)
: WebDriverException
Traceback (most recent call last):
File "/var/task/src/lambda_function.py", line 284, in lambda_handler
driver.find_element_by_xpath('//*[@id="searchForm"]/div[4]/ul/li[1]/input').click()
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 385, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 955, in find_element
'value': value})['value']
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "/var/task/lib/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
(Session info: headless chrome=65.0.3325.146)
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.9.85-38.58.amzn1.x86_64 x86_64)

This is the code leading up to the line that fails:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--window-size=1280x1696')
chrome_options.add_argument('--user-data-dir=/tmp/user-data')
chrome_options.add_argument('--hide-scrollbars')
chrome_options.add_argument('--enable-logging')
chrome_options.add_argument('--log-level=0')
chrome_options.add_argument('--v=99')
chrome_options.add_argument('--single-process')
chrome_options.add_argument('--data-path=/tmp/data-path')
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--homedir=/tmp')
chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
chrome_options.add_argument('user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36')
chrome_options.binary_location = '/var/task/bin/headless-chromium'

driver = webdriver.Chrome(chrome_options=chrome_options)
driver.implicitly_wait(3)
driver.get(URL)

Any advice would be fantastic.

@zekth

This comment has been minimized.

Copy link

commented Apr 19, 2018

I'm experiencing the same issue but using node 6.10. Using the chrome binary given with this project : https://github.com/blackboard/lambda-selenium
it works perfectly.
I'm using the same parameters as @robroc

@robroc

This comment has been minimized.

Copy link
Author

commented Apr 19, 2018

After a lot of tooling around, I got it to work on Lambda. The problem was with incompatible versions of serverless-chrome, chromedriver, and Selenium. These are the versions that play well together in Lambda. Why is beyond me:

chromedriver v.2.37
severless-chrome v.0.0-37
selenium 2.53.6 (for Python)

@zekth

This comment has been minimized.

Copy link

commented Apr 19, 2018

Thanks for the quick feeback @robroc . I'm going to update the chromedriver to 2.37 and check if it works.

@zekth

This comment has been minimized.

Copy link

commented Apr 19, 2018

@robroc it works 💃

Is there any document where we have the correlation between chromedriver and chrome itself?

Edit: Everything is written on the chromedriver page: https://sites.google.com/a/chromium.org/chromedriver/downloads

Maybe a warning on the documentation of serverless-chrome could be a good point for troubleshooting?

@rodel-talampas

This comment has been minimized.

Copy link

commented Jul 6, 2018

@robroc
You saved my day mate!!!

adieuadieu added a commit that referenced this issue Jul 8, 2018

docs: add a note about incompatible versions of selenium
Suggested in #133. Adds a note about incompatible versions of selenium, chromedriver
@adieuadieu

This comment has been minimized.

Copy link
Owner

commented Jul 8, 2018

Added a note to the docs.

@adieuadieu adieuadieu closed this Jul 8, 2018

@rneu31

This comment has been minimized.

Copy link

commented Jul 23, 2018

Did anyone look into investigating why we can't use newer versions of Chromium? Seems like pinning on such old versions works up until you want a new feature!

cc: @robroc @adieuadieu

@zekth

This comment has been minimized.

Copy link

commented Jul 23, 2018

This is related to the chromedriver itself

@rneu31

This comment has been minimized.

Copy link

commented Jul 23, 2018

Just for an additional data point, leaving chromedriver at 2.37 (which claims to support Chrome v64-66), and switching to v1.0.0-41 (Chromium 65.0.3282) of this project triggers the error. The version of selenium itself seems to have no impact (as suggested earlier).

@rneu31

This comment has been minimized.

Copy link

commented Jul 23, 2018

Found some time to do a little more digging. The only error in the chromedriver log that jumps out at me is

[1532377838.274][SEVERE]: CreatePlatformSocket() returned an error, errno=1: Operation not permitted (1)
[1532377838.274][INFO]: listen on IPv6 failed with error ERR_ACCESS_DENIED

Anyone smarter than I? Thanks!

@NikolaiT

This comment has been minimized.

Copy link

commented Sep 10, 2018

Can we now use newer versions of headless-chromium? Or still use the old version combination?

@zekth

This comment has been minimized.

Copy link

commented Sep 10, 2018

@NikolaiT versions of chromium/chromedriver are tied in term of compatibility look here : http://chromedriver.chromium.org/downloads

And check for the SUPPORT section of each chrome driver version to know which one fits your setup.

@marioavs

This comment has been minimized.

Copy link

commented May 22, 2019

Thank you @zekth for the reference. This combination of versions worked out:

chromedriver 2.43
severless-chrome 1.0.0-55
selenium 3.14 (Python package)

@robroc

This comment has been minimized.

Copy link
Author

commented May 22, 2019

@marioavs It looks like chromedriver versions below 2.46 are no longer offered. Do you know if those versions of serverless-chrome and Selenium work with the offered versions?

@marioavs

This comment has been minimized.

Copy link

commented May 22, 2019

chromedriver versions below 2.46 are still available, for example chromedriver version 2.43 that I mentioned.

Version 2.44 did not work for me, that is why I wanted to share the specific versions that did work in my tests. serverless-chrome 1.0.0-55 is built with chromium 69.0.3497.81 (stable channel) for amazonlinux:2017.03, that means it "should" work with chromedriver versions 2.41, 2.42, 2.43 and 2.44.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.