WebDriver is not working properly with pages after HTTP redirect in IE?

[lukeis]

Originally reported on Google Code with ID 232

Browser: IE
Version: 7
Operating System: WinXP SP2
WebDriver Java client version: 0.6.964

Example code:

    public static void main(String[] args) {
        InternetExplorerDriver webDriver = new InternetExplorerDriver();
        try {
            webDriver.get("http://apphost:8888/SecuredAction.jsp");
            // webDriver.get("http://www.google.com");
            WebElement inputUserName = webDriver.findElement(By.name
("username"));
            WebElement inputPassword = webDriver.findElement(By.name
("password"));
...
        } catch(Exception e) {
            e.printStackTrace();
        } finally {
            webDriver.quit();
        }
    }

The code above is working with error:

org.openqa.selenium.NoSuchElementException: Unable to find element by name 
using "username" (16) (no document found)
System info: os.name: 'Windows XP', os.arch: 'x86', os.version: '5.1', 
java.version: '1.6.0_03'
Driver info: driver.version: ie
    at org.openqa.selenium.ie.ErrorHandler.verifyErrorCode
(ErrorHandler.java:60)
    at org.openqa.selenium.ie.Finder.handleErrorCode(Finder.java:206)
    at org.openqa.selenium.ie.Finder.findElementByName(Finder.java:143)
    at org.openqa.selenium.By$4.findElement(By.java:146)
    at org.openqa.selenium.ie.Finder.findElement(Finder.java:194)
    at org.openqa.selenium.ie.InternetExplorerDriver.findElement
(InternetExplorerDriver.java:261)

When user is not logged in, then there will be redirect to other host with 
login page. Login page has "username" and "password" fields. Login page 
appears in the IE during this code execution.
Also the calls webDriver.getTitle() or webDriver.getPageSource() return 
empty strings, though the login page is showing in the browser.

But same code above for google.com site is working fine...
Code above is working fine for Firefox and HtmlUnit implementation

Reported by Krapivin914 on 2009-08-25 13:06:31

[lukeis]

I'm getting some similar stuff with Chrome.  I think it's that the browsers send page
loaded events differently.  I think that Firefox and HtmlUnit consider a redirect to
be one page loaded while IE (and maybe Chrome) consider it two pages (the redirector
then the redirect target.)

Reported by geis.ken on 2009-11-21 00:59:50

[lukeis]

It's generally difficult finding out when a page has finished loading - including 
following redirections (one would have to examine each page to see that it has no 
redirections, timers that are set, or anything else that might trigger event in the

future before declaring the page to be fully and completely loaded).

The common solution to this would be using the Wait class to wait until the elements

you wish to interact with appear on the page. This does not guarantee that the page

is fully loaded, just that the login element (for example) is there and it can be 
interacted with (Personally I think it's a good thing, as users may try to login 
before the page is entirely loaded, if they already see the username / password 
boxes). That's actually very similar to your findElement/catch loop, actually.

Reported by eran.mes on 2010-01-11 15:10:29

[lukeis]

Confirming this, just wasted a day on nailing a weird condition down to this root
cause (verified).. The InternetExplorerDriver() definitely appears to unexpectedly
return prematurely from e.g. a login button click, while the browser is still
following redirect(s, 5x HTTP 302 over here... urgh, I know!).  If the test does e.g.
an assert "fast enough" (just after the click) to find something on the page it
expects the browser to be on after an e.g. login, then it works sometimes, sometimes
not... the worst is that this is a "timing" issue - we didn't often see it on local
workstations, and then started running WebTest on faster integration servers where
the weird condition started appearing more regularly.  Autsch!

Working around it for now via an explicit loop with findElement/catch
(NoSuchElementException) sitting on its ass waiting around for what is expected on
the following page to really be in the browser... ugly.

> generally difficult finding out when a page has finished loading - including
following redirections (one would have to examine each page to see that it has no
redirections, timers that are set, or anything else that might trigger event in the
future before declaring the page to be fully and completely loaded).

Generally speaking, no problem with your statement, understood. - BUT speaking
specifically about the bug reported here, I disagree and would argue that it is fair
to expect WebDriver to handle an HTTP-level (code 302; NOT the ïn-page HTML META
header!) redirect "transparently", i.e. not returning until the, or like in our case,
several 302 redirects are done and the browser has ended up with the actual page
(HTTP 200).

The JavaDoc even explicitly says so, for WebDriver.get() : "... and the method will
block until the load is complete. This will follow redirects issued either by the
server [or as a meta-redirect from within the returned HTML]", nd for
WebElement.click() : "... this method will block until the page has loaded ...", but
apparently the InternetExplorerDriver does not behave as documented.

Reported by michael.vorburger on 2010-01-12 14:38:43

[lukeis]

Reported by jari.bakken on 2010-06-17 00:37:05

• Labels added: Component-WebDriver

[lukeis]

Hello everyone,

I know the original ticket is about IE and Java, but since others have reported the
same issue with Chrome I'd like to offer a slightly hacky solution for those who can't
wait for an official fix.

The issue with redirects and instant returns from operations was rendering the ChromeDriver
completely unuseable when used on sites that had any form of page load latency (i.e.
every non-local site in the world!). As has been mentioned above, working out when
a page is finished loading is a pain, and wait/spin functions are a really bad idea
(for example, this will fail if you have 2 pages each with a Continue button that you
click one after the order. It'll go click-click and only do 1 page transition). I think,
however, that I've found a decent way to identify when the page is finished, and moved
the wait code into the RemoteWebDriver class. It's working for me anyway for both page
redirects and normal page transitions, no extra waits in my actual test script code.

Instructions below are for C# and the ChromeDriver, but I'm sure it'll translate over
to Java easily enough:

Replace the content_script.js for the Chrome extension with the one attached to this
post. New stuff starts on line 11 and 132. The code for onsubmit may not even be neccessary,
I think the onbeforeunload is enough, but I've left it in.

Make some small changes to the source files:

DriverCommand.cs
Add another value to the DriverCommand enum called "GetCustomReadyState".

ChromeCommandExecutor.cs
In InitializeCommandNameMap(), add the following line:
    commandNameMap.Add(DriverCommand.GetCustomReadyState, "getCustomReadyState");

RemoteWebDriver.cs
Add the following code to the start of FindWebElement(string mechanism, string value)
and adjust polling as required:

  for (int i = 0; i < 60; i++)
  {
    Response getCustomReadyStateResponse = Execute(DriverCommand.GetCustomReadyState,
null);
    if ((string)getCustomReadyStateResponse.Value == "ready")
    {
      break;
    }
    System.Threading.Thread.Sleep(500);
  }

NOTE 1: obviously the new "getCustomReadyState" will only work on Chrome with these
instructions, so you may want to add a check for what sort of Driver class you are
currently using.
NOTE 2: I've put this code to check readystate in FindElement because 99% of the time
my interactions with the browser are "find element, perform action on it". I guess
if you need to check ready state before performing some other action you might need
this check elsewhere.

I'd be delighted to hear if anyone else has success with this.

Kind regards,

Rich

Reported by xiaofu.dev on 2010-06-28 08:04:46

---

- _Attachment: [content_script.js](https://storage.googleapis.com/google-code-attachments/selenium/issue-232/comment-5/content_script.js)_

[lukeis]

I also have problems with redirect, see

http://code.google.com/p/selenium/issues/detail?id=829

Reported by maggus.staab on 2010-09-24 11:32:06

[lukeis]

This is a major blocking issue for us, since our application is only supported on IE.
So this problem is preventing us from upgrading from Selenium RC 1.x. We are using
single sign-on with a CAS server, and the script is not recognizing the page has returned
after the redirect from the CAS server. However, the exact same script works perfectly
on the Firefox driver. For now we'll have to stick with the old RC until this issue
is resolved. If there is a way to bump the priority of this, please do so. thanks!
Jonathan 

Reported by jonhhuang on 2011-01-13 08:11:53

[lukeis]

The IE Driver has been rewritten, can you please retry this?

Reported by theautomatedtester on 2011-04-12 05:46:58

• Status changed: NeedsClarification
• Labels added: Browser-IE