Click button does not work after upgrade to selenium 2.48.0

[Tset-Noitamotua]

Doing test automation on a Ember.js web application:
I have a button which is covered by a <div> with a cooky notification. It looks something like this:

With Selenium 2.47.1 clicking on that button works as expected in Firefox.
With Selenium 2.48.0 clicking this button does not work in Firefox, although there is no exception or error. It's like selenium thinks that the click worked properly but actually the click had no effect. Here the Robot Framework log:

The structure of HTML is something like this:

<header>...</header>
<main>
  <div>
    ::before
    <form>
      <button>Last check</button>
    </form>
  ::after
  </div>
</main>
<div>
  <div class="cookie-notification">
    ::before
    <a href="http://...">Close</a>
    ::after
  </div>
</div>

Just for comparison - executing the same click with Selenium 2.47.1 in Chrome results in a WebDriverException:

Traceback (most recent call last):
  File "<string>", line 2, in click_button
  File "C:\Python27\lib\site-packages\Selenium2Library\keywords\keywordgroup.py", line 15, in _run_on_failure_decorator
    return method(*args, **kwargs)
  File "C:\Python27\lib\site-packages\Selenium2Library\keywords\_formelement.py", line 316, in click_button
    element.click()
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py", line 69, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py", line 448, in _execute
    return self._parent.execute(command, params)
  File "C:\Python27\lib\site-packages\Selenium2Library\webdrivermonkeypatches.py", line 11, in execute
    result = self._base_execute(driver_command, params)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 196, in execute
    self.error_handler.check_response(response)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 181, in check_response
    raise exception_class(message, screen, stacktrace)

System configuration I:

• Microsoft Windows 7 64 bit
• Firefox 41.0.2
• Selenium 2.47.1
• selenium2library 1.7.4
• Robot Framework 2.9.2
  ==> click <button> works

System configuration II:

• same as configuration I but
• Google Ghrome Version 46.0.2490.80 m
  ==> click <button> does not work, WebDriverException

System configuration III:

• same as configuration I but
• Selenium 2.48.0
  ==> click <button> does not work, no error/exception

[davehunt]

This is due to the change 0eec81d, which makes Selenium more closely resemble a user when interacting with elements. You could try scrolling the element into view before attempting to click it. /cc @barancev

[costag1982]

I had the exact same issue yesterday and I did indeed try scrolling to the element first before clicking and it still didn't work. I changed my webdriver to Chrome and the test passed which then led me to believe this issue was created in the recent update.

[Tset-Noitamotua]

There is no possibility to scroll down in my case. My solution was to downgrade to Selenium 2.47.1

[barancev]

I wonder how a real user can click this button if it can't be scrolled into view?

[vveliev]

I think I have a similar issue ...
#1166

[Tset-Noitamotua]

@barancev
Real user can close the cooky notification by clicking the X on the right. The button is then uncovered and can be clicked. For my automated test I need to implement extra logic to handle this.

[Tset-Noitamotua]

@davehunt
Thanks for the hint on 0eec81d. Does that mean that by me descriped behavier is intended and should be understood as a feature?

If so, then I would suggest to give the user a notification or exception like the one of Chrome webdriver: Element is not clickable at point (x, y). Other element would receive the click: <div class="classname">...</div>. This would make it easier for the user to understand what happened.

[phoenix384]

This is no feature and it made us roll back to 2.47 also.
The argument "simulating what the user can do" is overused to the detriment of ease of use.
Depending on the overlapping element a real user could just scroll the page a bit or close the overlapping element. At least the scrolling would be something FirefoxDriver should be doing automatically after this change.
Or how should I guess beforehand when a page is accidently scrolled to the wrong position?

This behaviour in ChromeDriver is annoying, but in Firefox it is now even worse since it simply clicks and nothing happens instead of at least throwing an exception like ChromeDriver.
With webelement.click() I expect the click to go to the WebElement and not to the position of the WebElement. If I explicitly want to click its position I can achieve that with the Actions class.

If you make such changes you should provide a possibility to disable this new behaviour for compatibility purposes.

[barancev]

If you want to "click" an element that is hidden behind some other element you can use JavascriptExecutor. It is close to impossible to implement autoscrolling that could resolve overlapping.

[malligr8]

Hi,

Element is not clickable at (x,y) co-ordinate seems to be an old Issue
2766, which we have solution. I have the same problem and i tried below
solution.

((JavascriptExecutor)
driver).executeScript("arguments[0].scrollIntoView(true);", element);

More reliable solution would be

Waitfor the Element to untill it was stable by any of the
Expectedconditions and then do scroll on weblement as below.
((JavascriptExecutor)
driver).executeScript("arguments[0].scrollIntoView(true);", element);

Regards
Malli

On Fri, Oct 30, 2015 at 8:28 PM, Alexei Barantsev notifications@github.com
wrote:

If you want to "click" an element that is hidden behind some other element
you can use JavascriptExecutor. It is close to impossible to implement
autoscrolling that could resolve overlapping.

—
Reply to this email directly or view it on GitHub
#1202 (comment)
.

[davehunt]

I'm also seeing this, where an element is temporarily over the target element. It took too long for me to confirm exactly what was going on (although I suspected it almost immediately). The patch in #1211 helped with my confirmation. In my case, I've decided that the proper solution would be for the application under test to give a suitable indication that the obstructing element is not displayed. Due to the initial state of this element not being displayed, there's no appropriate waits that I can use - I even experimented with executing JavaScript with a MutationObserver. The project in question is not actively being developed, so I'll open an issue against it, but likely just add a sleep to my test suite. If anyone's interested, here's how to reproduce my issue:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
wait = WebDriverWait(driver, 10)
driver.get("http://beta.123done.org/")
main_window = driver.current_window_handle
for i in range(5):
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#signinhere button"))).click()
    wait.until(lambda d: len(d.window_handles) > 1)
    driver.switch_to.window(list(filter(lambda w: w != main_window, driver.window_handles))[0])
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#authentication_email"))).send_keys("webqabrowserid@aol.com")
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "p.submit button.isStart"))).click()
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#authentication_password"))).send_keys("dsadadawdsda")
    submit = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "p.submit button.isTransitionToSecondary"))).click()
    assert wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#cannot_authenticate"))).text == "Password incorrect"
    driver.close()
    driver.switch_to.window(main_window)
driver.quit()

Thanks to @p0deje for his patch in #1211 and also to @barancev who was able to confirm that he saw this issue whilst I was investigating it.

[TraceyC77]

I have also observed this behavior in 2.28.2, which is compounded by Selenium scrolling an element too far, making it appear underneath a page's floating header (such as with a navigation bar). I was able to reproduce the bug with the page http://www.redkitetechnologies.com/services/

This bug makes 2.28.2 unusable in my company.

Expected

The element which was specified by xpath is clicked on

Actual

The element is scrolled behind the page header. The location where the element was scrolled to is clicked. This results on a click on the page header, instead of the element which was specified.

This simple perl script will reproduce the error (adjust things for your environment)

#!/usr/local/bin/perl

use warnings;
use strict;
use Utils::WebDriver;
use Test::More tests => 1;

my $webdriver = Utils::WebDriver->new(browser_url => "http://www.redkitetechnologies.com/services/" );
$webdriver->set_window_size( 768, 1024 );

$webdriver->find('//[@id="post-22"]/div/section/div[3]/article/a/h2', 'xpath')->click;
note ("Clicked the Solutions title and its page should be loaded");
like (
$webdriver->find( '//[@id="post-695"]/div/div/h1', 'xpath' )->get_text,
qr/solutions/,
"Correct page loaded after clicking Solutions"
);

$webdriver->quit();
exit(0);

[davehunt]

@TraceyC77 Selenium will scroll an element before interacting with it if it's not in view, and will align it with the top of the viewport. You could prevent this by proactively scrolling the element into view by using execute_script with some JavaScript such as: arguments[0].scrollIntoView(false); to align with the bottom of the viewport, or arguments[0].scrollIntoView(); window.scrollBy(0, 120); to align to the top and then scroll by the height of the header.

[phoenix384]

We have nearly 3000 tests running and none of the tests failing with 2.48 because of this change detects a defect or something that has to be changed in the application under test.
So for us this change just causes work. And it's even worse since no error is thrown in case of a "wrong click".
Again the request: please make it possible to disable this new behaviour.

[barancev]

PR #1211 merged, that makes Firefox to behave like Chrome (and unlike IE that clicks the topmost element and does not throw).

But I'll keep this issue open because I'm going to implement a capability to choose to desired behavior -- to throw (default) or to click the topmost.

[kpakur]

Interesting, I have a very similar case that may be related to this. Chrome driver, c#.

There is a form and a buton under the form. This used to work fine, I guess it was scrolling automatically somehow to the button at the bottom of the screen and the buton could be clicked. But since a small addition to the form, where a new input is added dynamicaly based on some other inputs on the form, this stopped working and cannot find the buton at the bottom of the page. I just guess that maybe there is some calculation of the height of the page or elements positions for the view port, when the new input is dynamicaly added by javascript this extends the height of entire page, and then the buton goes below the original size and cannot be found.

[ronwsmith]

+1 to fix or be able to disable.

Very frustrating issue to track down until I found this thread.

In our case, we have a floating header that is clickable. As forms are filled out, the page scrolls down. When attempting to click on Submit, it is behind the header and the change introduced in 0eec81d makes it click on the header and send the user back to the home page (most frequently).

For quite a while, we thought there were application issues and we were being redirected home because of an error that didn't log anything.

[ronwsmith]

@barancev, the ideal case would be neither of your suggestions in #1202 (comment). The browser should scroll until the desired element is clickable just like it does with filling out form fields, etc.

PR #1211 and your ideas work if it's impossible to scroll to get to the desired element (e.g. some overlay that needs to be closed first).

[barancev]

@ronwsmith, it is not always possible to make overlapped elements clickable by scrolling. I understand that you mean fixed position blocks, but it is only one of possible cases.

[ronwsmith]

Thanks for the clarification. I think we're on the same page here.

[doberkofler]

@barancev We seem to have the same issue after upgrading from 2.47 to 2.48 with a huge number of tests that started failing. I still do not understand what exactly causes the tests to fail and what needs to be done to make them work again? We do use a fixed page header but I do not understand what needs to be changed to make our tests work again or is switching back to 2.47 the only option for now? Does this issue affect any page element outside the visible area or is it only related to overlapping elements? Any help or clarification is appreciated!

[molsky]

@doberkofler We are also having same kind of issues with RF and S2L combination. Moved back to 2.47.3 because didn't have time investigate why some of the test cases started to fail with this 2.48.0. 2.47.3 can find those buttons and even screenshots shows that button is visible but for some reason 2.48.0 fails with them.

[barancev]

Anyone having a fixed position header can set capability elementScrollBehavior to 1 that makes Selenium to scroll elements to the bottom (default is 0 that means scrolling to the top).

[barancev]

The original issue reported in this thread is much trickier. Selenium thinks that element is in view, so it does not scroll at all. And the element remains overlapped.

I'm trying to figure out a sensible solution for suck cases. May be we should check if the element is overlapped by a fixed position element, and if this happens we have to try scrolling a bit in the "opposite" direction. But this "solution" drives me crazy...

[HardenDev]

I totally get it now, thanks for all you guys' help.

[phoenix384]

elementScrollBehavior = 1 solves the problem with the fixed header only if the element is completely outside the current view. If it is already covered by the fixed header, it is not scrolled to the bottom...

[gterre]

I've created a utility class that brings an element into view in the middle of the viewport if it's obstructed by another element. Works with chrome and firefox latest drivers, should work with IE but I haven't tested it. Use it to deal with those nasty header / footers or anything of that matter.

see https://github.com/gterre/stuff/blob/master/JavascriptUtils.java

usage... JavascriptUtils.bringIntoView(WebElement element);

[Tset-Noitamotua]

and the 100th comment goes to ... the author of the post: Tset-Noitamotua!

Sorry for spamming, guys. I´m just having a nice day .. hope you all, too! ;-)

[sab39]

I'm having a particularly tricky case of this problem: the thing covering up the element to be clicked on is the tooltip that Firefox puts in the bottom-left/bottom-right of the viewport indicating the destination of a link. For a real user, this moves back and forth to the left and right to avoid the mouse, but Selenium's mouse simulation doesn't seem to trigger this same thing, so the tooltip stays in the bottom-right even though that's where the element to be clicked is.

(Is there any reason for there not to be an elementScrollBehavior.Middle? Seems less likely to be problematic than either top OR bottom)

[jimevans]

@sab39 To answer your parenthetical question at the end of your comment, the reason is that the JavaScript scrollIntoView function only supports the two values for top and bottom. There is no "scroll to middle" functionality provided by that function[1].

[1] For the nitpickers, yes, I'm aware there's a proposed overload to scrollIntoView that allows a JSON object as its argument instead of a boolean, but that overload is not widely implemented yet.

[keithrz]

So, is this in the nodejs driver yet? If not, where is the code for the nodejs driver ? (Just wrapping my head around where I can get/implement a fix for my protractor setup, thanks!!)

[nielei3]

2.51.0 does not fix this issue. I have to downgrade to 2.47.1.

[JulienBreton]

2.53.0 + capabilities.setCapability("elementScrollBehavior", 1); solve the issue in my case.

[Ukrainis]

I have the same problem, using Selenium with C#+Specflow. All configuration is in the app.config file. Does anyone know, how to add capabilities in app.config file?
Thanks in advance.
Valentin.

[perwez123]

I have upgraded the selenium to 2.53 java version . Now click fucntion does not work for some of the scenario .

It throwing the following error

Element is not clickable at point (1195.1500244140625, 618). Other element would receive the click: Next (WARNING: The server did not provide any stacktrace information)

I used all the way commented for this issue but no luck . This issue occurring all the browser .

When used the javascript function for click event then it works fine , but for some the page it does not return the desired result .

Can anyone tell where the problem exactly ?

[perwez123]

I have attached the screen shot for the button i am trying to click and also the element locators . This issue have stopped my entire script from execution . I cant find any solution to fix this issue . Need immediate attention please.

Screen shot attached below

[jianfengs]

@perwez123 instead of using click, call javascript directly. In your case, it will be "onNext('intro')".

[perwez123]

Hi ,

javaScript works for this case , but when moving to next page clicking on
Next button without javascript call, then there are few more button
appears. But if i use the javascript directly then it does not display all
the button . Its behaving very strange. I have no idea why all button does
not display when using javascript directly.

Help is appreciated .

Thanks

On Fri, Apr 15, 2016 at 9:03 PM, jianfengs notifications@github.com wrote:

@perwez123 https://github.com/perwez123 instead of using click, call
javascript directly. In your case, it will be "onNext('intro')".

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1202 (comment)

Thanks & Regards

• Md Perwez Alam*

[perwez123]

Can you please send me the complete code for javascript call as per my screen shared ?

[perwez123]

Please find the screen shot below for the scenario where i faced the problem.

Expected behavior - When i click on Next button then it should move to the next page and all the button displayed should display .

Actual behavior - The check answers button does not display when clicking on next button using javascript call directly

Screen shot

[perwez123]

javascript executor does not return the expected result , As when click fucntion is triggered ,it seems that main page is navigated forward but the footer navigation part did not move to next page . It seems there is problem in javascript executor .

[cadet-blue-git]

I have this problem extensively when the topmost element passes a click event down to the desired element. Is there any way to make selenium click the topmost element and throw a warning instead of throwing an error?

[lukeis]

@Cyberia-- the ability to disable the error in firefox was added in 2.52 with this ab99406 see the first file changed for the capability name to disable it.

Since this issue is closed and consistently attracting new commentors, i'm going to lock it. Please log new issues with reproducible test cases if you have a bug. If you are asking a question or need a clarification on something, please do not log a new issue and email the selenium-users google group.