New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid does not handle w3c capabilities correctly #3808

Closed
digiology opened this Issue Apr 11, 2017 · 76 comments

Comments

Projects
None yet
@digiology

digiology commented Apr 11, 2017

Meta -

OS: Mac OS El Capitan 10.11.6

Selenium Version: 3.3.3

Browser: Firefox

Browser Version: 52.0.2 (64-bit)

Expected Behavior -

In python, the following should create a Remote driver:

from selenium import webdriver

cap = {'platform': 'ANY', 'browserName': 'firefox', 'version': '', 'marionette': True, 'javascriptEnabled': True}
driver = webdriver.Remote(command_executor='http://<my_server_ip>:4444/wd/hub', desired_capabilities=cap)

(note: this works with selenium 2.53.6 with python2.7 when executed against the same server)

Actual Behavior -

Thows exception: KeyError: 'message'

When I pdb into this I see the following response from the server:

{'value': '{"status":13,"value":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putMapEntries","lineNumber":500},{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putAll","lineNumber":784},{"fileName":"DesiredCapabilities.java","className":"org.openqa.selenium.remote.DesiredCapabilities","methodName":"\u003cinit\u003e","lineNumber":54},{"fileName":"RequestHandler.java","className":"org.openqa.grid.web.servlet.handler.RequestHandler","methodName":"process","lineNumber":104},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"process","lineNumber":83},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"doPost","lineNumber":67},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":707},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":790},{"fileName":"ServletHolder.java","className":"org.seleniumhq.jetty9.servlet.ServletHolder","methodName":"handle","lineNumber":841},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doHandle","lineNumber":543},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doHandle","lineNumber":1584},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doHandle","lineNumber":1228},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":168},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doScope","lineNumber":481},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doScope","lineNumber":1553},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":166},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doScope","lineNumber":1130},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"handle","lineNumber":141},{"fileName":"HandlerWrapper.java","className":"org.seleniumhq.jetty9.server.handler.HandlerWrapper","methodName":"handle","lineNumber":132},{"fileName":"Server.java","className":"org.seleniumhq.jetty9.server.Server","methodName":"handle","lineNumber":564},{"fileName":"HttpChannel.java","className":"org.seleniumhq.jetty9.server.HttpChannel","methodName":"handle","lineNumber":320},{"fileName":"HttpConnection.java","className":"org.seleniumhq.jetty9.server.HttpConnection","methodName":"onFillable","lineNumber":251},{"fileName":"AbstractConnection.java","className":"org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback","methodName":"succeeded","lineNumber":279},{"fileName":"FillInterest.java","className":"org.seleniumhq.jetty9.io.FillInterest","methodName":"fillable","lineNumber":112},{"fileName":"ChannelEndPoint.java","className":"org.seleniumhq.jetty9.io.ChannelEndPoint$2","methodName":"run","lineNumber":124},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool","methodName":"runJob","lineNumber":672},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2","methodName":"run","lineNumber":590},{"fileName":"Thread.java","className":"java.lang.Thread","methodName":"run","lineNumber":745}]}}', 'status': 500}

Steps to reproduce -

Execute the above steps in python using selenium 3.3.3, firefox 52.0.2, geckodriver 0.15.0. The server is using selenium-server-standalone-3.3.1.jar on ubuntu 16.10. I'm happy to provide the ip of the server privately if it'll help.

@digiology

This comment has been minimized.

digiology commented Apr 11, 2017

Also, testing with curl seems to work fine:

curl -v -X POST http://<my_server_ip>:4444/wd/hub/session -d '{"desiredCapabilities":{"browserName":"firefox", "marionette": "true", "javascriptEnabled": "true"}}'

curl -v 'http://<my_server_ip>:4444/wd/hub/session/SESSION_ID'

@digiology

This comment has been minimized.

digiology commented Apr 11, 2017

Ok, looks it actually works if I change the port to 5555.

I'm guessing that's connecting directly to a node rather than the hub? Why would port 4444 work with selenium 2.5 but not 3.3.3?

@foxleoly

This comment has been minimized.

foxleoly commented Apr 11, 2017

I met the same issue. and Using the same command on selenium 3.3.0 working fine. Now I'm downgrade to 3.3.0.

In [1]: from selenium.webdriver import *

In [2]: dri = Remote('http://my_sever_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

KeyError Traceback (most recent call last)
in ()
----> 1 dri = Remote('http://my_server_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in init(self, command_executor, desired_capabilities, browser_profile, proxy, keep_alive, file_detector)
96 warnings.warn("Please use FirefoxOptions to set browser profile",
97 DeprecationWarning)
---> 98 self.start_session(desired_capabilities, browser_profile)
99 self._switch_to = SwitchTo(self)
100 self._mobile = Mobile(self)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in start_session(self, capabilities, browser_profile)
183 parameters = {"capabilities": w3c_caps,
184 "desiredCapabilities": capabilities}
--> 185 response = self.execute(Command.NEW_SESSION, parameters)
186 if 'sessionId' not in response:
187 response = response['value']

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
247 response = self.command_executor.execute(driver_command, params)
248 if response:
--> 249 self.error_handler.check_response(response)
250 response['value'] = self._unwrap_value(
251 response.get('value', None))

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
108 value = message
109 try:
--> 110 message = message['message']
111 except TypeError:
112 message = None

KeyError: 'message'

In [3]: exit

@rgonalo

This comment has been minimized.

rgonalo commented Apr 11, 2017

I have the same issue with Selenium python client 3.3.2 or 3.3.3 against a Selenium server 3.3.1. It works with Selenium python client 3.3.1.

The problem has been introduced with commit abe1f45. It uses the new W3C start session (send "capabilities" instead of "requiredCapabilities"), but it seems that Selenium 3.3.1 server still needs "requiredCapabilities".

@jaiaravindj

This comment has been minimized.

jaiaravindj commented Apr 14, 2017

I have downgraded selenium, selenium standalone server and it works fine.

@lmtierney lmtierney added C-py C-server and removed C-server labels Apr 19, 2017

@gennadykr

This comment has been minimized.

gennadykr commented Apr 19, 2017

I can create Remote driver if run server locally as
java -jar selenium-server-standalone-3.3.1.jar

But it fails for a local grid started as:

java -jar selenium-server-standalone-3.3.1.jar -role hub
java -jar selenium-server-standalone-3.3.1.jar -role node

I found that this grid works only with "selenium-server-standalone-2.53.1.jar", but do not even with the first 3.x one "selenium-server-standalone-3.0.0.jar"

I use python 3.6, selenium 3.3.3, and command

 wd = webdriver.Remote("http://192.168.50.30:4444/wd/hub",desired_capabilities=DesiredCapabilities.CHROME)
@smccarthy

This comment has been minimized.

smccarthy commented Apr 21, 2017

Here is a full stacktrace using the python bindings 3.4.0 with selenium server (in grid hub/node mode) : https://gist.github.com/smccarthy/6778f2af97b99e301767438de01c62fb

I tried with chromedriver 2.28 and 2.29 , but I seem to get the same issue with DesiredCapabilities.FIREFOX

@lmtierney lmtierney added C-grid and removed C-py labels Apr 21, 2017

@lmtierney

This comment has been minimized.

Member

lmtierney commented Apr 21, 2017

This appears to be a grid issue, it does not like 'capabilities' being in the parameters for new session

@lmtierney

This comment has been minimized.

Member

lmtierney commented Apr 21, 2017

Or rather that it expects desiredCapabilities to be inside capabilities

return new JsonToBeanConverter().convert(Map.class, map.getAsJsonObject("capabilities").getAsJsonObject("desiredCapabilities"));

@lmtierney lmtierney changed the title from Creating new Remote driver works in selenium 2.5 but not with 3.3.3 to Grid does not handle w3c capabilities correctly Apr 22, 2017

@lmtierney

This comment has been minimized.

Member

lmtierney commented Apr 22, 2017

@shs96c you'd probably be best at handling the fix for this

@mhomb

This comment has been minimized.

mhomb commented Apr 28, 2017

Is there any update on this? I am still stuck on 3.0.1 because of this bug.

@bjharo

This comment has been minimized.

bjharo commented Apr 28, 2017

I'm having the same issue using the .Net bindings on the client side.

OS: Windows 10
Selenium Version: 3.4
Browser: Chrome
Browser Version: 57
Grid Version (Hub & Node): 3.4

I was testing this setup with a locally running hub and node on my dev laptop before upgrading our actual grid. Anytime I request a browser directly from the hub, I get an error. This is not a problem using 3.3 nor is it a problem using 3.4 when I bypass the grid and use a local browser.

@lmtierney

This comment has been minimized.

Member

lmtierney commented Apr 29, 2017

@bjharo see @jimevans's explanation here #3929 (comment)

@jimevans

This comment has been minimized.

Member

jimevans commented Apr 30, 2017

And the reason it's showing up now is that 3.4 is the first release to consistently attempt to send the W3C spec-compliant payload for a new session command (containing a capabilities property) across the wire. This is where the bug is in the hub code.

@shs96c shs96c self-assigned this May 2, 2017

@sajidchingchong

This comment has been minimized.

sajidchingchong commented May 4, 2017

Do we have any work around of it?

@jakeczyz

This comment has been minimized.

jakeczyz commented May 8, 2017

I also am seeing this problem. My workaround in Python has been to force an earlier version of selenium when installing. I.e., pip3 uninstall selenium && pip3 install selenium==3.3.1

@Grimlek

This comment has been minimized.

Grimlek commented May 8, 2017

As a workaround this is what worked for me....go into the selenium webdriver client

I am using python client

go into the remote/webdriver.py

you will see this within the start_session(some args) method

parameters = {"capabilities": w3c_caps,
                      "desiredCapabilities": capabilities}

remove the "capabilities": w3c_caps

so you end up with:

parameters = { "desiredCapabilities": capabilities}

I am sure that this is very similar to the other client bindings. Just a temporary workaround until this can be fixed.

@barancev

This comment has been minimized.

Member

barancev commented Aug 11, 2017

Should be fixed in version 3.5, please reopen the issue if it's not true.

@barancev barancev closed this Aug 11, 2017

@cristian-sobral-fluke

This comment has been minimized.

cristian-sobral-fluke commented Aug 15, 2017

Not working for me in version 3.5. I had to downgrade to v3.3

@barancev

This comment has been minimized.

Member

barancev commented Aug 15, 2017

@cristiansw "Not working" is a zero info message. What browser, what capabilities, what error appears?

@fatdh

This comment has been minimized.

fatdh commented Aug 16, 2017

v3.5.1 fixes the issue for C#/Chrome/Windows
@cristiansw, have you upgraded both your project's Selenium and the selenium-server-standalone jar?

Thanks

@cristian-sobral-fluke

This comment has been minimized.

cristian-sobral-fluke commented Aug 16, 2017

@fatdh thanks for the update, I've just upgraded to 3.5.1, however, I'm using selenium grid running on docker and the docker image is still using selenium-server-standalone 3.4.0

@barancev Let's backtrack a little bit.. I wouldn't say that

Should be fixed in version 3.5

is very informing.. was it fixed or not? I was just trying to help & signal that the original issue might need to be reopened.

@dado21780

This comment has been minimized.

dado21780 commented Aug 16, 2017

Would it possible to get the version 3.5 of the java selenium libraries on the central maven repository?
Thanks!

@firstchild78

This comment has been minimized.

firstchild78 commented Aug 17, 2017

I can verify that the 3.5.1 fixed the issue for me.(C#) I'm also using Selenium Grid v3.4.0
Thanks.

@mpgo13

This comment has been minimized.

mpgo13 commented Aug 17, 2017

Should this be fixed in the standalone mode as well?

I'm getting the following (using FF54 and geckodriver 0.18.0):

selenium.common.exceptions.WebDriverException: Message: mouseMoveTo
Build info: version: '3.5.0', revision: '8def36e068', time: '2017-08-10T23:00:22.093Z'
System info: host: 'myhost', ip: '130.194.77.215', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.6', java.version: '1.8.0_101'
Driver info: driver.version: unknown
Stacktrace:
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:222)
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:119)
    at org.openqa.selenium.remote.server.ProtocolConverter.handle (ProtocolConverter.java:72)
    at org.openqa.selenium.remote.server.ServicedSession.execute (ServicedSession.java:81)
    at org.openqa.selenium.remote.server.WebDriverServlet.lambda$handle$0 (WebDriverServlet.java:231)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    at java.lang.Thread.run (Thread.java:745)
@mpgo13

This comment has been minimized.

mpgo13 commented Aug 17, 2017

Sorry, my fault, it works now :)

@manoj9788

This comment has been minimized.

Member

manoj9788 commented Aug 17, 2017

I think we can close this issue.

@barancev

This comment has been minimized.

Member

barancev commented Aug 17, 2017

FYI: version 3.5.1 is available in maven central

@timbru31

This comment has been minimized.

timbru31 commented Aug 22, 2017

@mpgo13

Sorry, my fault, it works now :)

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

edit:
note to future self: read comments above
note to other readers: pass -enablePassThrough to the standalone

@mpgo13

This comment has been minimized.

mpgo13 commented Aug 22, 2017

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

I was upgrading the server while still using old python selenium.

@gwtuser

This comment has been minimized.

gwtuser commented Sep 8, 2017

I was redirected to this issue report by:
"UnsupportedCommandException mouseMoveTo with 3.4 on firefox with geckodriver 0.16.1 #4008"

FF 52.3.0
geckodriver 18
Selenium 3.5.3

I am not using a standalone, so how can I pass "-enablePassthrough"? I'm just creating FirefoxDriver(), and specifying the location of the geckodriver via System.setProperty("webdriver.gecko.driver", ...) in Java.
I don't want to use "legacy" mode (which is not supported in FF55+, right?), but I also don't want to have to switch to RemoteWebDriver.

Is using FirefoxDriver + gecko a supported path forward?

@usmonster

This comment has been minimized.

usmonster commented Sep 8, 2017

FYI, -enablePassThrough requires a boolean argument and is enabled by default since v3.5.0. On my side, I had to set -enablePassThrough false on my nodes (same with standalone) to get things working with FF55+ & geckodriver 18.

@gwtuser

This comment has been minimized.

gwtuser commented Sep 8, 2017

Yeah, I'm aware it requires an argument. I was trying to figure where to apply that option, since I'm not running any nodes, nor a standalone Selenium server. I'm launching WebDriver by calling new FirefoxDriver() in Java, and then driver.get() on a URL. Perhaps the redirect from issue #4008 was incorrect, as this issue refers to "Grid" and I'm not using that, nor standalone WebDriver.

@codingphil

This comment has been minimized.

codingphil commented Sep 15, 2017

Is the Pass Through Mode the way going forward for supporting W3C compliant webdrivers in the standalone server or is it planned to make it working also without Pass Through Mode?

@shs96c

This comment has been minimized.

Member

shs96c commented Sep 15, 2017

@DaryaOrlova

This comment has been minimized.

DaryaOrlova commented Sep 20, 2017

Issue still reproduces for me (Ruby v2.3.3). I'm using Selenium Grid v3.3.1 running on docker and selenium-webdriver v3.5.2 on local machine.

the error is:

undefined method `capabilities' for nil:NilClass (NoMethodError)
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/w3c/capabilities.rb:154:in `from_oss'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:183:in `merged_capabilities'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:103:in `create_session'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:54:in `handshake'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/driver.rb:39:in `initialize'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `new'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `for'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver.rb:86:in `for'
/Users/odachka/aytm/aytmRegressionTests/pvRegressionTests/features/support/hooks.rb:12:in `<top (required)>'
@lmtierney

This comment has been minimized.

Member

lmtierney commented Sep 20, 2017

@DaryaOrlova that's because you're using grid v3.3.1, this was fixed in the grid code not the ruby gem

@tkueck

This comment has been minimized.

tkueck commented Sep 26, 2017

So we've been running into this at work with the C# bindings. By making these changes starting at line 1108 in RemoteWebDriver.cs we are now capable of using desired capabilities as worked prior to the w3c changes. https://gist.github.com/tkueck/178eb16058c9e3b240107dcc2c4cb2f3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment