Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid does not handle w3c capabilities correctly #3808

Closed
ghost opened this issue Apr 11, 2017 · 76 comments
Closed

Grid does not handle w3c capabilities correctly #3808

ghost opened this issue Apr 11, 2017 · 76 comments
Assignees
Labels

Comments

@ghost
Copy link

@ghost ghost commented Apr 11, 2017

Meta -

OS: Mac OS El Capitan 10.11.6

Selenium Version: 3.3.3

Browser: Firefox

Browser Version: 52.0.2 (64-bit)

Expected Behavior -

In python, the following should create a Remote driver:

from selenium import webdriver

cap = {'platform': 'ANY', 'browserName': 'firefox', 'version': '', 'marionette': True, 'javascriptEnabled': True}
driver = webdriver.Remote(command_executor='http://<my_server_ip>:4444/wd/hub', desired_capabilities=cap)

(note: this works with selenium 2.53.6 with python2.7 when executed against the same server)

Actual Behavior -

Thows exception: KeyError: 'message'

When I pdb into this I see the following response from the server:

{'value': '{"status":13,"value":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putMapEntries","lineNumber":500},{"fileName":"HashMap.java","className":"java.util.HashMap","methodName":"putAll","lineNumber":784},{"fileName":"DesiredCapabilities.java","className":"org.openqa.selenium.remote.DesiredCapabilities","methodName":"\u003cinit\u003e","lineNumber":54},{"fileName":"RequestHandler.java","className":"org.openqa.grid.web.servlet.handler.RequestHandler","methodName":"process","lineNumber":104},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"process","lineNumber":83},{"fileName":"DriverServlet.java","className":"org.openqa.grid.web.servlet.DriverServlet","methodName":"doPost","lineNumber":67},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":707},{"fileName":"HttpServlet.java","className":"javax.servlet.http.HttpServlet","methodName":"service","lineNumber":790},{"fileName":"ServletHolder.java","className":"org.seleniumhq.jetty9.servlet.ServletHolder","methodName":"handle","lineNumber":841},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doHandle","lineNumber":543},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doHandle","lineNumber":1584},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextHandle","lineNumber":188},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doHandle","lineNumber":1228},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":168},{"fileName":"ServletHandler.java","className":"org.seleniumhq.jetty9.servlet.ServletHandler","methodName":"doScope","lineNumber":481},{"fileName":"SessionHandler.java","className":"org.seleniumhq.jetty9.server.session.SessionHandler","methodName":"doScope","lineNumber":1553},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"nextScope","lineNumber":166},{"fileName":"ContextHandler.java","className":"org.seleniumhq.jetty9.server.handler.ContextHandler","methodName":"doScope","lineNumber":1130},{"fileName":"ScopedHandler.java","className":"org.seleniumhq.jetty9.server.handler.ScopedHandler","methodName":"handle","lineNumber":141},{"fileName":"HandlerWrapper.java","className":"org.seleniumhq.jetty9.server.handler.HandlerWrapper","methodName":"handle","lineNumber":132},{"fileName":"Server.java","className":"org.seleniumhq.jetty9.server.Server","methodName":"handle","lineNumber":564},{"fileName":"HttpChannel.java","className":"org.seleniumhq.jetty9.server.HttpChannel","methodName":"handle","lineNumber":320},{"fileName":"HttpConnection.java","className":"org.seleniumhq.jetty9.server.HttpConnection","methodName":"onFillable","lineNumber":251},{"fileName":"AbstractConnection.java","className":"org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback","methodName":"succeeded","lineNumber":279},{"fileName":"FillInterest.java","className":"org.seleniumhq.jetty9.io.FillInterest","methodName":"fillable","lineNumber":112},{"fileName":"ChannelEndPoint.java","className":"org.seleniumhq.jetty9.io.ChannelEndPoint$2","methodName":"run","lineNumber":124},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool","methodName":"runJob","lineNumber":672},{"fileName":"QueuedThreadPool.java","className":"org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2","methodName":"run","lineNumber":590},{"fileName":"Thread.java","className":"java.lang.Thread","methodName":"run","lineNumber":745}]}}', 'status': 500}

Steps to reproduce -

Execute the above steps in python using selenium 3.3.3, firefox 52.0.2, geckodriver 0.15.0. The server is using selenium-server-standalone-3.3.1.jar on ubuntu 16.10. I'm happy to provide the ip of the server privately if it'll help.

@ghost
Copy link
Author

@ghost ghost commented Apr 11, 2017

Also, testing with curl seems to work fine:

curl -v -X POST http://<my_server_ip>:4444/wd/hub/session -d '{"desiredCapabilities":{"browserName":"firefox", "marionette": "true", "javascriptEnabled": "true"}}'

curl -v 'http://<my_server_ip>:4444/wd/hub/session/SESSION_ID'

@ghost
Copy link
Author

@ghost ghost commented Apr 11, 2017

Ok, looks it actually works if I change the port to 5555.

I'm guessing that's connecting directly to a node rather than the hub? Why would port 4444 work with selenium 2.5 but not 3.3.3?

@foxleoly
Copy link

@foxleoly foxleoly commented Apr 11, 2017

I met the same issue. and Using the same command on selenium 3.3.0 working fine. Now I'm downgrade to 3.3.0.

In [1]: from selenium.webdriver import *

In [2]: dri = Remote('http://my_sever_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

KeyError Traceback (most recent call last)
in ()
----> 1 dri = Remote('http://my_server_ip:4444/wd/hub', DesiredCapabilities.HTMLUNIT)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in init(self, command_executor, desired_capabilities, browser_profile, proxy, keep_alive, file_detector)
96 warnings.warn("Please use FirefoxOptions to set browser profile",
97 DeprecationWarning)
---> 98 self.start_session(desired_capabilities, browser_profile)
99 self._switch_to = SwitchTo(self)
100 self._mobile = Mobile(self)

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in start_session(self, capabilities, browser_profile)
183 parameters = {"capabilities": w3c_caps,
184 "desiredCapabilities": capabilities}
--> 185 response = self.execute(Command.NEW_SESSION, parameters)
186 if 'sessionId' not in response:
187 response = response['value']

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
247 response = self.command_executor.execute(driver_command, params)
248 if response:
--> 249 self.error_handler.check_response(response)
250 response['value'] = self._unwrap_value(
251 response.get('value', None))

/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
108 value = message
109 try:
--> 110 message = message['message']
111 except TypeError:
112 message = None

KeyError: 'message'

In [3]: exit

@rgonalo
Copy link

@rgonalo rgonalo commented Apr 11, 2017

I have the same issue with Selenium python client 3.3.2 or 3.3.3 against a Selenium server 3.3.1. It works with Selenium python client 3.3.1.

The problem has been introduced with commit abe1f45. It uses the new W3C start session (send "capabilities" instead of "requiredCapabilities"), but it seems that Selenium 3.3.1 server still needs "requiredCapabilities".

@jaiaravindj
Copy link

@jaiaravindj jaiaravindj commented Apr 14, 2017

I have downgraded selenium, selenium standalone server and it works fine.

@gennadykr
Copy link

@gennadykr gennadykr commented Apr 19, 2017

I can create Remote driver if run server locally as
java -jar selenium-server-standalone-3.3.1.jar

But it fails for a local grid started as:

java -jar selenium-server-standalone-3.3.1.jar -role hub
java -jar selenium-server-standalone-3.3.1.jar -role node

I found that this grid works only with "selenium-server-standalone-2.53.1.jar", but do not even with the first 3.x one "selenium-server-standalone-3.0.0.jar"

I use python 3.6, selenium 3.3.3, and command

 wd = webdriver.Remote("http://192.168.50.30:4444/wd/hub",desired_capabilities=DesiredCapabilities.CHROME)
@smccarthy
Copy link

@smccarthy smccarthy commented Apr 21, 2017

Here is a full stacktrace using the python bindings 3.4.0 with selenium server (in grid hub/node mode) : https://gist.github.com/smccarthy/6778f2af97b99e301767438de01c62fb

I tried with chromedriver 2.28 and 2.29 , but I seem to get the same issue with DesiredCapabilities.FIREFOX

@lmtierney lmtierney added C-grid and removed C-py labels Apr 21, 2017
@lmtierney
Copy link
Member

@lmtierney lmtierney commented Apr 21, 2017

This appears to be a grid issue, it does not like 'capabilities' being in the parameters for new session

@lmtierney
Copy link
Member

@lmtierney lmtierney commented Apr 21, 2017

Or rather that it expects desiredCapabilities to be inside capabilities

return new JsonToBeanConverter().convert(Map.class, map.getAsJsonObject("capabilities").getAsJsonObject("desiredCapabilities"));

@lmtierney lmtierney changed the title Creating new Remote driver works in selenium 2.5 but not with 3.3.3 Grid does not handle w3c capabilities correctly Apr 22, 2017
@lmtierney
Copy link
Member

@lmtierney lmtierney commented Apr 22, 2017

@shs96c you'd probably be best at handling the fix for this

@mhomb
Copy link

@mhomb mhomb commented Apr 28, 2017

Is there any update on this? I am still stuck on 3.0.1 because of this bug.

@bjharo
Copy link

@bjharo bjharo commented Apr 28, 2017

I'm having the same issue using the .Net bindings on the client side.

OS: Windows 10
Selenium Version: 3.4
Browser: Chrome
Browser Version: 57
Grid Version (Hub & Node): 3.4

I was testing this setup with a locally running hub and node on my dev laptop before upgrading our actual grid. Anytime I request a browser directly from the hub, I get an error. This is not a problem using 3.3 nor is it a problem using 3.4 when I bypass the grid and use a local browser.

@lmtierney
Copy link
Member

@lmtierney lmtierney commented Apr 29, 2017

@bjharo see @jimevans's explanation here #3929 (comment)

@jimevans
Copy link
Member

@jimevans jimevans commented Apr 30, 2017

And the reason it's showing up now is that 3.4 is the first release to consistently attempt to send the W3C spec-compliant payload for a new session command (containing a capabilities property) across the wire. This is where the bug is in the hub code.

@shs96c shs96c self-assigned this May 2, 2017
@sajidchingchong
Copy link

@sajidchingchong sajidchingchong commented May 4, 2017

Do we have any work around of it?

@jakeczyz
Copy link

@jakeczyz jakeczyz commented May 8, 2017

I also am seeing this problem. My workaround in Python has been to force an earlier version of selenium when installing. I.e., pip3 uninstall selenium && pip3 install selenium==3.3.1

@Grimlek
Copy link

@Grimlek Grimlek commented May 8, 2017

As a workaround this is what worked for me....go into the selenium webdriver client

I am using python client

go into the remote/webdriver.py

you will see this within the start_session(some args) method

parameters = {"capabilities": w3c_caps,
                      "desiredCapabilities": capabilities}

remove the "capabilities": w3c_caps

so you end up with:

parameters = { "desiredCapabilities": capabilities}

I am sure that this is very similar to the other client bindings. Just a temporary workaround until this can be fixed.

@barancev
Copy link
Member

@barancev barancev commented Aug 11, 2017

Should be fixed in version 3.5, please reopen the issue if it's not true.

@barancev barancev closed this Aug 11, 2017
@cristian-sobral-fluke
Copy link

@cristian-sobral-fluke cristian-sobral-fluke commented Aug 15, 2017

Not working for me in version 3.5. I had to downgrade to v3.3

@barancev
Copy link
Member

@barancev barancev commented Aug 15, 2017

@CristianSw "Not working" is a zero info message. What browser, what capabilities, what error appears?

@wildroo
Copy link

@wildroo wildroo commented Aug 16, 2017

v3.5.1 fixes the issue for C#/Chrome/Windows
@CristianSw, have you upgraded both your project's Selenium and the selenium-server-standalone jar?

Thanks

@cristian-sobral-fluke
Copy link

@cristian-sobral-fluke cristian-sobral-fluke commented Aug 16, 2017

@fatdh thanks for the update, I've just upgraded to 3.5.1, however, I'm using selenium grid running on docker and the docker image is still using selenium-server-standalone 3.4.0

@barancev Let's backtrack a little bit.. I wouldn't say that

Should be fixed in version 3.5

is very informing.. was it fixed or not? I was just trying to help & signal that the original issue might need to be reopened.

@dado21780
Copy link

@dado21780 dado21780 commented Aug 16, 2017

Would it possible to get the version 3.5 of the java selenium libraries on the central maven repository?
Thanks!

@firstchild78
Copy link

@firstchild78 firstchild78 commented Aug 17, 2017

I can verify that the 3.5.1 fixed the issue for me.(C#) I'm also using Selenium Grid v3.4.0
Thanks.

@mpgo13
Copy link

@mpgo13 mpgo13 commented Aug 17, 2017

Should this be fixed in the standalone mode as well?

I'm getting the following (using FF54 and geckodriver 0.18.0):

selenium.common.exceptions.WebDriverException: Message: mouseMoveTo
Build info: version: '3.5.0', revision: '8def36e068', time: '2017-08-10T23:00:22.093Z'
System info: host: 'myhost', ip: '130.194.77.215', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.6', java.version: '1.8.0_101'
Driver info: driver.version: unknown
Stacktrace:
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:222)
    at org.openqa.selenium.remote.http.AbstractHttpCommandCodec.encode (AbstractHttpCommandCodec.java:119)
    at org.openqa.selenium.remote.server.ProtocolConverter.handle (ProtocolConverter.java:72)
    at org.openqa.selenium.remote.server.ServicedSession.execute (ServicedSession.java:81)
    at org.openqa.selenium.remote.server.WebDriverServlet.lambda$handle$0 (WebDriverServlet.java:231)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    at java.lang.Thread.run (Thread.java:745)
@mpgo13
Copy link

@mpgo13 mpgo13 commented Aug 17, 2017

Sorry, my fault, it works now :)

@manoj9788
Copy link
Member

@manoj9788 manoj9788 commented Aug 17, 2017

I think we can close this issue.

@barancev
Copy link
Member

@barancev barancev commented Aug 17, 2017

FYI: version 3.5.1 is available in maven central

@timbru31
Copy link

@timbru31 timbru31 commented Aug 22, 2017

@mpgo13

Sorry, my fault, it works now :)

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

edit:
note to future self: read comments above
note to other readers: pass -enablePassThrough to the standalone

@mpgo13
Copy link

@mpgo13 mpgo13 commented Aug 22, 2017

Can you share what your fix is/was? Experiencing the same issue with v3.5.2 and geckodriver v0.18 and hoped that v3.5.0 should fix this.

I was upgrading the server while still using old python selenium.

@gwtuser
Copy link

@gwtuser gwtuser commented Sep 8, 2017

I was redirected to this issue report by:
"UnsupportedCommandException mouseMoveTo with 3.4 on firefox with geckodriver 0.16.1 #4008"

FF 52.3.0
geckodriver 18
Selenium 3.5.3

I am not using a standalone, so how can I pass "-enablePassthrough"? I'm just creating FirefoxDriver(), and specifying the location of the geckodriver via System.setProperty("webdriver.gecko.driver", ...) in Java.
I don't want to use "legacy" mode (which is not supported in FF55+, right?), but I also don't want to have to switch to RemoteWebDriver.

Is using FirefoxDriver + gecko a supported path forward?

@usmonster
Copy link

@usmonster usmonster commented Sep 8, 2017

FYI, -enablePassThrough requires a boolean argument and is enabled by default since v3.5.0. On my side, I had to set -enablePassThrough false on my nodes (same with standalone) to get things working with FF55+ & geckodriver 18.

@gwtuser
Copy link

@gwtuser gwtuser commented Sep 8, 2017

Yeah, I'm aware it requires an argument. I was trying to figure where to apply that option, since I'm not running any nodes, nor a standalone Selenium server. I'm launching WebDriver by calling new FirefoxDriver() in Java, and then driver.get() on a URL. Perhaps the redirect from issue #4008 was incorrect, as this issue refers to "Grid" and I'm not using that, nor standalone WebDriver.

@codingphil
Copy link

@codingphil codingphil commented Sep 15, 2017

Is the Pass Through Mode the way going forward for supporting W3C compliant webdrivers in the standalone server or is it planned to make it working also without Pass Through Mode?

@shs96c
Copy link
Member

@shs96c shs96c commented Sep 15, 2017

@DaryaOrlova
Copy link

@DaryaOrlova DaryaOrlova commented Sep 20, 2017

Issue still reproduces for me (Ruby v2.3.3). I'm using Selenium Grid v3.3.1 running on docker and selenium-webdriver v3.5.2 on local machine.

the error is:

undefined method `capabilities' for nil:NilClass (NoMethodError)
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/w3c/capabilities.rb:154:in `from_oss'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:183:in `merged_capabilities'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:103:in `create_session'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/bridge.rb:54:in `handshake'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/remote/driver.rb:39:in `initialize'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `new'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver/common/driver.rb:58:in `for'
/Users/odachka/.rvm/gems/ruby-2.3.3/gems/selenium-webdriver-3.5.2/lib/selenium/webdriver.rb:86:in `for'
/Users/odachka/aytm/aytmRegressionTests/pvRegressionTests/features/support/hooks.rb:12:in `<top (required)>'
@lmtierney
Copy link
Member

@lmtierney lmtierney commented Sep 20, 2017

@DaryaOrlova that's because you're using grid v3.3.1, this was fixed in the grid code not the ruby gem

@tkueck
Copy link

@tkueck tkueck commented Sep 26, 2017

So we've been running into this at work with the C# bindings. By making these changes starting at line 1108 in RemoteWebDriver.cs we are now capable of using desired capabilities as worked prior to the w3c changes. https://gist.github.com/tkueck/178eb16058c9e3b240107dcc2c4cb2f3

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet