Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upSelenium does not wait after calling WebElement.submit() #4570
Comments
mgruner
referenced this issue
Aug 24, 2017
Closed
Selenium does not wait after calling WebElement.submit() #878
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 24, 2017
Member
For issues please provide a concise reproducible test case and describe what results you are seeing and what results you expect.
See CONTRIBUTING.md
|
For issues please provide a concise reproducible test case and describe what results you are seeing and what results you expect. See CONTRIBUTING.md |
barancev
added
the
R-awaiting answer
label
Aug 24, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Aug 28, 2017
@barancev of course.
You need the CPAN module Selenium::Remote::Driver, installable via cpanm Selenium::Remote::Driver, for example.
Here's a small Perl test script:
use strict;
use warnings;
use Selenium::Remote::Driver;
my $Selenium = Selenium::Remote::Driver->new(
remote_server_addr => 'localhost',
port => '4444',
browser_name => 'firefox',
platform => 'ANY',
extra_capabilities => {
marionette => \0, # Required to run FF 47 or older on Selenium 3+.
},
);
$Selenium->get('https://www.google.com');
my $Element = $Selenium->find_element('form input.gsfi', 'css');
$Element->send_keys('Selenium');
$Element->submit();
if ($Selenium->get_page_source() !~ 'seleniumhq\.org') {
die "Error: Selenium home page not found.";
}
else {
print "Ok: Selenium home page found.\n"
}
If you run this with Firefox 47 (and the marionette => 0 line enabled), it works correctly. After the submit, Selenium waits, and get_page_source retrieves the new page with the Selenium home page in the results.
Tested with Firefox 55.0.3 (and the marionette => 0line disabled), Selenium does not wait. After the submit, get_page_source delivers the original page with the search form and no results.
mgruner
commented
Aug 28, 2017
|
@barancev of course. You need the CPAN module Here's a small Perl test script: If you run this with Firefox 47 (and the Tested with Firefox 55.0.3 (and the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 28, 2017
Member
Last Firefox version supported by legacy Firefox driver is 52 ESR.
To run tests in Firefox 55 you have to use geckodriver.
|
Last Firefox version supported by legacy Firefox driver is 52 ESR. |
barancev
closed this
Aug 28, 2017
barancev
removed
the
R-awaiting answer
label
Aug 28, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Aug 28, 2017
@barancev Please don't close this! Of course I used geckodriver. Removing the line marionette => \0 from the configuration causes Selenium to use the installed geckodriver to talk to Firefox. And that seems to cause the issue of the wrong waiting. I even originally submitted the issue with the geckodriver project, but they told me to report it here instead, as they believe it could be an issue with the shim that Selenium uses to implement submitElement with WebDriver, that does not have such a function.
mgruner
commented
Aug 28, 2017
|
@barancev Please don't close this! Of course I used geckodriver. Removing the line |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 28, 2017
Member
Sorry, I saw marionette => \0 in your code and decided you're trying to drive Firefox 55 with legacy driver.
But I have to say that CPAN module Selenium::Remote::Driver is not a software developed by Selenium team, so you have to report the issue to this Perl library bug tracker.
|
Sorry, I saw But I have to say that CPAN module |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 28, 2017
Member
One more note: I tried to run your code sample with Selenium::Remote::Driver version 1.0 and Selenium server 3.5.2 and it fails on send_keys operation because the client library does not send element identifier.
Client side log:
Use of uninitialized value in substitution iterator at C:/Perl64/site/lib/Seleni
um/Remote/Commands.pm line 437.
Error while executing command: sendKeysToElement: The requested resource could n
ot be found, or a request was received using an HTTP method that is not supporte
d by the mapped resource.: POST /session/dfb48504-8f12-4ae3-a31f-da8c2984e84d/el
ement//value
Build info: version: '3.5.2', revision: 'unknown', time: '2017-08-24T16:43:06.09
4Z'
System info: host: 'ALEXEI-PC', ip: '192.168.56.1', os.name: 'Windows 7', os.arc
h: 'amd64', os.version: '6.1', java.version: '1.8.0_144'
Driver info: driver.version: unknown at C:/Perl64/site/lib/Selenium/Remote/Drive
r.pm line 313.
at C:/Perl64/site/lib/Selenium/Remote/Driver.pm line 313.
Server log:
14:43:12.687 INFO - /session: Executing POST on /session (handler: BeginSession)
14:43:12.688 INFO - Capabilities are: Capabilities {acceptSslCerts=true, browser
Name=firefox, javascriptEnabled=true, platformName=ANY, version=, platform=ANY,
}
14:43:12.688 INFO - Capabilities {acceptSslCerts=true, browserName=firefox, java
scriptEnabled=true, platformName=ANY, version=, platform=ANY, } matched class or
g.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.se
lenium.firefox.GeckoDriverService)
14:43:12.689 INFO - Capabilities {acceptSslCerts=true, browserName=firefox, java
scriptEnabled=true, platformName=ANY, version=, platform=ANY, } matched class or
g.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.se
lenium.chrome.ChromeDriverService)
1503920592712 geckodriver INFO geckodriver 0.18.0
1503920592719 geckodriver INFO Listening on 127.0.0.1:20430
1503920593222 geckodriver::marionette INFO Starting browser C:\Program File
s\Nightly\firefox.exe with args ["-marionette"]
1503920593664 addons.xpi WARN Error parsing extensions state: [Excepti
on... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [am
IAddonManagerStartup.readStartupData]" nsresult: "0x80520012 (NS_ERROR_FILE_NOT
_FOUND)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :
: loadExtensionState :: line 1521" data: no] Stack trace: loadExtensionState()@
resource://gre/modules/addons/XPIProvider.jsm:1521 < getInstallState()@resource:
//gre/modules/addons/XPIProvider.jsm:1556 < checkForChanges()@resource://gre/mod
ules/addons/XPIProvider.jsm:3091 < startup()@resource://gre/modules/addons/XPIPr
ovider.jsm:2158 < callProvider()@resource://gre/modules/AddonManager.jsm:263 < _
startProvider()@resource://gre/modules/AddonManager.jsm:731 < startup()@resource
://gre/modules/AddonManager.jsm:898 < startup()@resource://gre/modules/AddonMana
ger.jsm:3082 < observe()@jar:file:///C:/Program%20Files/Nightly/omni.ja!/compone
nts/addonManager.js:65
1503920594015 Marionette INFO Enabled via --marionette
Unable to read VR Path Registry from C:\Users\alexei\AppData\Local\openvr\openvr
paths.vrpath
[Parent 8804] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src
/chrome/common/ipc_channel_win.cc, line 346
Unable to read VR Path Registry from C:\Users\alexei\AppData\Local\openvr\openvr
paths.vrpath
[Child 17904] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src
/chrome/common/ipc_channel_win.cc, line 346
[Child 17904] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src
/chrome/common/ipc_channel_win.cc, line 346
1503920595641 Marionette INFO Listening on port 65223
1503920595983 Marionette DEBUG Register listener.js for window 42949672
97
14:43:16.003 INFO - Found handler: org.openqa.selenium.remote.server.ServicedSes
sion@1251f217
14:43:16.004 INFO - Handler thread for session bda92b59-5509-4186-b151-0d13d877c
745 (firefox): Executing POST on /session/bda92b59-5509-4186-b151-0d13d877c745/u
rl (handler: ServicedSession)
1503920596012 Marionette DEBUG Received DOM event "beforeunload" for "a
bout:blank"
1503920596313 Marionette DEBUG Received DOM event "pagehide" for "about
:blank"
1503920596314 Marionette DEBUG Received DOM event "unload" for "about:b
lank"
1503920596385 Marionette DEBUG Received DOM event "DOMContentLoaded" fo
r "https://www.google.ru/?gfe_rd=cr&ei=0wGkWfOlItHCtAGmmKbgAw"
1503920596581 Marionette DEBUG Received DOM event "pageshow" for "https
://www.google.ru/?gfe_rd=cr&ei=0wGkWfOlItHCtAGmmKbgAw"
14:43:16.592 INFO - Found handler: org.openqa.selenium.remote.server.ServicedSes
sion@1251f217
14:43:16.592 INFO - Handler thread for session bda92b59-5509-4186-b151-0d13d877c
745 (firefox): Executing POST on /session/bda92b59-5509-4186-b151-0d13d877c745/e
lement (handler: ServicedSession)
14:43:16.727 INFO - Found handler: org.openqa.selenium.remote.server.ServicedSes
sion@1251f217
14:43:16.728 INFO - Handler thread for session bda92b59-5509-4186-b151-0d13d877c
745 (firefox): Executing POST on /session/bda92b59-5509-4186-b151-0d13d877c745/e
lement//value (handler: ServicedSession)
14:43:16.737 INFO - Found handler: org.openqa.selenium.remote.server.ServicedSes
sion@1251f217
|
One more note: I tried to run your code sample with Client side log: Server log: |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Aug 28, 2017
@barancev what Selenium::Remote::Driver does is just to send the submitElement command to Selenium, no matter which browser (version) is being used. Therefore I don't believe that the problem exists in the Perl bindings. Do you think you could try this very simple snipped in another language that is supported by Selenium, or am I required to provide a transpilation? Which languages would be acceptable?
mgruner
commented
Aug 28, 2017
|
@barancev what |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Aug 28, 2017
@barancev, please use a recent version of Selenium::Remote::Driver, 1.20 is current.
mgruner
commented
Aug 28, 2017
|
@barancev, please use a recent version of |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 28, 2017
Member
If Perl binding just sends submitElement commant -- it's problem of the binding. Because W3C WebDriver standard does not include this command: https://w3c.github.io/webdriver/webdriver-spec.html
That's exactly what @andreastt pointed at. This command should be either emulated in the client library, or removed at all. W3C conformant drivers can't handle this command.
|
If Perl binding just sends That's exactly what @andreastt pointed at. This command should be either emulated in the client library, or removed at all. W3C conformant drivers can't handle this command. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Aug 28, 2017
@barancev just to make sure that I understand this correctly. Selenium does receive the command, and also causes a submit on the page. Just that it does not wait correctly, as it did before using the WebDriver interface (with old Firefox and also with current Chrome). So it seems that the "emulation" you mentioned does exist in Selenium, right? Shouldn't it be fixed there then?
mgruner
commented
Aug 28, 2017
|
@barancev just to make sure that I understand this correctly. Selenium does receive the command, and also causes a submit on the page. Just that it does not wait correctly, as it did before using the WebDriver interface (with old Firefox and also with current Chrome). So it seems that the "emulation" you mentioned does exist in Selenium, right? Shouldn't it be fixed there then? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
commented
Aug 28, 2017
|
@andreastt explicitly mentioned it as a shim in Selenium that should be fixed. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Aug 28, 2017
Member
OK, I've figured out, that Selenium server actually provides such a shim for non-W3C-compliant client libraries. Sorry to mislead you.
Now back to the original issue. Should Selenium wait for the next page to load after submit operation or not? @shs96c ?
|
OK, I've figured out, that Selenium server actually provides such a shim for non-W3C-compliant client libraries. Sorry to mislead you. Now back to the original issue. Should Selenium wait for the next page to load after submit operation or not? @shs96c ? |
barancev
reopened this
Aug 28, 2017
barancev
added
A-needs decision
C-remote
labels
Aug 28, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
commented
Aug 28, 2017
|
@barancev NP. Thanks for responding so fast! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
commented
Sep 1, 2017
|
Selenium 3.5.3 has the same behaviour. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
commented
Sep 6, 2017
|
@barancev may I kindly ask for the status of this? How can we proceed? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
barancev
Sep 6, 2017
Member
No estimations. We don't consider this to be a critical issue. Proper use of WebDriverWait eliminates issues like this one.
|
No estimations. We don't consider this to be a critical issue. Proper use of WebDriverWait eliminates issues like this one. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Sep 6, 2017
@barancev to my knowledge, that is not possible from the Perl bindings. And submitElement is a part of the official Selenium API, isn't it?
mgruner
commented
Sep 6, 2017
|
@barancev to my knowledge, that is not possible from the Perl bindings. And |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Sep 6, 2017
Alternatively, WebDriver has the convenience method “submit” on every element. If you call this on an element within a form, WebDriver will walk up the DOM until it finds the enclosing form and then calls submit on that. (http://www.seleniumhq.org/docs/03_webdriver.jsp#selenium-webdriver-api-commands-and-operations)
mgruner
commented
Sep 6, 2017
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
shs96c
Sep 6, 2017
Member
Figuring out when the page load starts is a tricky problem. Since the submit is done using a pile of JS code, it's safe to assume that the JS has finished executing once the call returns, and we expect that the page loads have started too. The processing model of selenium means that the next request should wait until the current page is at the readyState expected by its page loading strategy.
Put another way, the drivers do a "best effort" to figure out if the page is going to reload, and the drivers also wait until they believe it's safe to process a command, but there's no way to guarantee that we successfully catch all the places where loads start. That's why most language bindings have the equivalent of the Wait class --- to handle just this kind of case.
For reference, "submit" is implemented as a JS shim. This simply walks up the DOM until it finds the FORM element, and then calls "submit()" on that. If you "click" on the submit button (as @andreastt implied) then you'll get the w3c behaviour, which includes a set of browser side checks to try and determine whether a page load will happen.
|
Figuring out when the page load starts is a tricky problem. Since the submit is done using a pile of JS code, it's safe to assume that the JS has finished executing once the call returns, and we expect that the page loads have started too. The processing model of selenium means that the next request should wait until the current page is at the Put another way, the drivers do a "best effort" to figure out if the page is going to reload, and the drivers also wait until they believe it's safe to process a command, but there's no way to guarantee that we successfully catch all the places where loads start. That's why most language bindings have the equivalent of the For reference, "submit" is implemented as a JS shim. This simply walks up the DOM until it finds the FORM element, and then calls "submit()" on that. If you "click" on the submit button (as @andreastt implied) then you'll get the w3c behaviour, which includes a set of browser side checks to try and determine whether a page load will happen. |
shs96c
closed this
Sep 6, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Sep 6, 2017
@shs96c thank you very much for the clarification. Unfortunately, document.readyState is complete right after calling submitElement (just tested with FF 55.0.3). I don't believe that is intended, is it? We have no problem in waiting for the right readyState, but with the current situation there is no way of detecting that the load even started. That's precisely what this issue is about.
Is there anything else we can try to work around this?
mgruner
commented
Sep 6, 2017
|
@shs96c thank you very much for the clarification. Unfortunately, |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
shs96c
Sep 6, 2017
Member
The readyState is something that browser updates --- it's nothing to do with selenium, though we do rely on its value.
The best solution is to make use of the Wait class, like so:
Wait<WebDriver> wait = new WebDriverWait(driver, 10, TimeUnit.SECONDS);
wait.until(driver -> { /* Your check here */ });
There are some pre-prepared conditions for waiting in the ExpectedConditions class.
|
The The best solution is to make use of the Wait class, like so: There are some pre-prepared conditions for waiting in the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
Sep 6, 2017
@shs96c we already have a waiting strategy in place for submits. Essentially we wait for the page being loaded completely and all our code being executed, which is verified by accessing a JavaScript variable that our code sets at the end. This would include document.readyState verification. Unfortunately, this waiting fails now because right after the submit() it reports that the page has loaded.
So what else should we wait for?
mgruner
commented
Sep 6, 2017
|
@shs96c we already have a waiting strategy in place for submits. Essentially we wait for the page being loaded completely and all our code being executed, which is verified by accessing a JavaScript variable that our code sets at the end. This would include |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
luke-hill
Sep 6, 2017
Contributor
On our site we have a late_loading_css hash and the page title. Once both pass we're happy.
|
On our site we have a late_loading_css hash and the page title. Once both pass we're happy. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jimevans
Sep 6, 2017
Member
@mgruner If it's that big a deal, wait for the element you're calling submit() on to become stale (which guarantees the page reload has started), then fall into your "wait for page load" routine.
|
@mgruner If it's that big a deal, wait for the element you're calling |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
shs96c
Sep 6, 2017
Member
@mgruner, unset the variable indicating the page is loaded before executing the submit.
|
@mgruner, unset the variable indicating the page is loaded before executing the submit. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
mgruner
commented
Sep 7, 2017
|
Thanks for all suggestions. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
harvesh123
Oct 12, 2017
Hi I'm using selenium 3.6.0
FF -55.0
geckodriver v0.19.0
I'm also having same problem i'm running 2 test cases one by one but first test cases need time to realod but ff doesn't wait and start second test cases. can any one suggest.....please
harvesh123
commented
Oct 12, 2017
|
Hi I'm using selenium 3.6.0 I'm also having same problem i'm running 2 test cases one by one but first test cases need time to realod but ff doesn't wait and start second test cases. can any one suggest.....please |
mgruner commentedAug 24, 2017
Meta -
OS: OS X 10.11.6 (15G1611)
Selenium Version: 3.4.0
Browser: Firefox 55.0.2
Originally opened at mozilla/geckodriver#878, but then I was referred to here.
Expected Behavior -
With Firefox 47 (without marionette), calling submit() on a WebElement caused Selenium to wait until the page load starts (just as it also works in current Chrome).
Actual Behavior -
This is not the case with current Firefox that uses WebDriver. Is that intended behaviour? Our test suite relies on that implicit wait currently.
Steps to reproduce -
If a submit() is triggered on a form, the following test code runs in the context of the old page.
Possible issue -
Information by @andreastt in the geckodriver issue:
Any help will be most appreciated. Sorry, I'm not familiar with the technical details of Selenium's internals.