Back Navigation partially broken in Firefox WebDriver

[lukeis]

Originally reported on Google Code with ID 3611

Platform:  Java 1.6.0_26 .. 1.6.0_31
OS:  Windows 7 64-bit
Firefox:  10 & 11
IE:  9
Selenium & RemoteDriver executing on same machine.


There exists some difference in back navigation on Firefox WebDriver via Java mechanisms
like
driver.navigate().back() or 
driver.executeScript("history.go(-1);", new Object[0]); 

That changed somewhere after Selenium 2.13.0 and can be demonstrated in 2.19.0 and
2.20.0.  I haven't bothered to narrow down any versions in between.

The change has something to do with the behavior of caching of pages rendered from
a POST operation.  Such that if you go to some arbitrary website, submit a form, see
the resulting page, navigate away, then navigate back you will now get a Document Expired
warning within Firefox under WebDriver.  Outside of Firefox WebDriver this warning
will not occur, not even with IE WebDriver.

For example:  
using regular Firefox or IE, go to 
http://www.txlottery.org/opencms/Games/Scratch_Offs/Retailer_Locator.jsp

Select a city and submit.  Navigate away from that page, then navigate
back in your browser.  No problem.  Now spawn an instance of Firefox
or IE via Selenium WebDriver and try it.  Firefox will report the
Document Expired error.  IE will not report any problem. 

This will not occur in regular Firefox, regular IE, or WebDriver IE.  So this is a
difference in FF browser behavior brought on by some change in Selenium.  Using an
existing, real user profile does not solve the problem so it cannot be some kind of
profile preference setting.

I've used the FireBug extension within FF to monitor the network request/response interactions
when doing this type of navigation.  As suspected, BACK will load from cache.  For
whatever reason FF WebDriver will not cache it, and it is unclear how to re-enable
that behavior.

I've tried examining commits to look for anything suspicious and the best I could come
up with is: 

r15354  "DanielWagnerHall for yurodivuie: Don't allow caching of GET
commands" (http://code.google.com/p/selenium/source/detail?r=15354&path=/trunk/java/client/src/org/openqa/selenium/remote/HttpCommandExecutor.java)


The code looks as if GO_BACK commands are represented as internal POST methods though,
but I'm not sure how that might affect a potential GET from cache.  I've reverted that
change locally and it didn't fix the problem...

The only clue I have to go on is that FF WebDriver's warning message (see screenshot).

Maybe org.openqa.selenium.remote.server.DriverServlet?  A change committed r15330 on
12/31/11 (which is consistent with the timeline of when Selenium was known to be working
and known to be broken 2.13 v.s 2.19)
    // Make sure our browser-clients never cache responses.
    response.setHeader("Expires", "Thu, 01 Jan 1970 00:00:00 GMT");
    response.setHeader("Cache-Control", "no-cache");

The commit message neglects any mention of a change to cache control.

...So maybe somewhere there was a change that is marking the document as already expired
or something even when it wouldn't ordinarily be. 

I realize there is debate about the appropriateness in caching of POST responses and
what not, however, the point here is that WebDriver is not emulating normal Firefox
behavior.

In my case everything was working fine under FF10 & Selenium 2.13.0 but unfortunately
FF11 auto-installed and is not working with Selenium 2.13.0.  I've stayed away from
modern Selenium versions because of this problem, but now I have no choice.

Reported by bap0679 on 2012-03-22 20:16:21

---

- _Attachment: SeleniumBackNavigationDocExpired.png
_

[lukeis]

Reported by barancev on 2012-03-27 05:47:38

• Labels added: Component-WebDriver, Browser-Firefox

[lukeis]

We have the same problem with FF10 und Selenium 2.20.0

Reported by erooolkoc on 2012-03-29 08:57:58

[lukeis]

I've tried reverting the change to cache-control introduced in DriverServlet r15330
& recompiling, but I still have the problem.  From what I gather from the selenium-users
group, python users aren't having a problem with FF11, Selenium 2.20 and WebDriver.
 So I'm still looking for a Java-specific problem.

Reported by bap0679 on 2012-03-29 18:50:24

[lukeis]

Using a suggestion by Svilen Ivanov (http://groups.google.com/group/selenium-users/msg/79f9d815e599ce95?&q=back)
I was able to re-use the Selenium 2.13 by manually editing install.rdf inside of webdriver.xpi
contained within the .jar file to allow it to work with FF11. 

Selenium 2.20 is still broken, but at least I have a workaround.  2.13 works for me
again.

Reported by bap0679 on 2012-03-31 08:47:35

[lukeis]

I'm seen this on 2.21 also.

Reported by emmanicolau on 2012-05-07 21:09:44

[lukeis]

Also Noticed in 2.21

Reported by andrew.mckee on 2012-05-21 13:59:06

[lukeis]

We are seeing an issue with back() navigation with Selenium 2.25 and Firefox 14. Any
updates on this fix? is it going to be part of 2.26? Chrome and IE don't seem to have
a problem, and the problem does not happen when running interactively without using
Selenium.

Reported by binod80 on 2012-10-23 15:23:40

[lukeis]

This also happens with Firefox 3.6. Using Firebug I could see that the first POST that
is sent as a result of a back operation, is blocked, only when using Selenium + Firefox.
Since it also happens with Firefox3.6, this likely cannot be a Firefox upgrade issue.

Reported by binod80 on 2012-11-02 16:03:22

[lukeis]

I switched to htmlunit 2.11 after hitting this with firefox 17.  Same problem.  Using
Selenium 2.25 java.

Reported by jashar@alumni.cmu.edu on 2012-11-29 10:38:13

[lukeis]

this was my very dodgy, very site-specific workaround:

                        // Necessary so we can come back to this page
                        // http://code.google.com/p/selenium/issues/detail?id=3611
                        String script = "var form_array = document.getElementsByTagName('form');"
+
                                        "for (var i=0, len=form_array.length; i < len;
i++) {" +
                                        "    form_array[i].setAttribute('method', 'get');"
+
                                        "}";
                        driver.executeScript(script);
                        driver.findElement(...).click();

Reported by jashar@alumni.cmu.edu on 2012-11-29 23:51:15

[lukeis]

I find this problem in FF10 with WebDriver 2.19. When I checked for the differences
in firefox preferrences settings using about:config, I could figure out that toggling
"offline-apps.allow_by_default" config removes the expired page issue. But since this
property is frozen, this can't be set in FirefoxProfile.

Reported by anand.v.nath on 2013-02-11 08:47:11

[lukeis]

There is an issue with Firefox regarding document expired. Issue is caused by sites
instructing Firefox not to cache the page. When you go back to the page, Firefox honors
the site's instructions and requires you to regenerate the request. When it's a POST
request, this requires an extra confirmation.
This might be related to that:-
https://bugzilla.mozilla.org/show_bug.cgi?id=200208
https://bugzilla.mozilla.org/show_bug.cgi?id=888757

Reported by roughplayer999 on 2013-09-16 10:15:54

[lukeis]

No this isn't because of a FireFox bug.  Several have stated that FireFox without WebDriver
does not have this problem.  FireFox with WebDriver with non-Java (Python) didn't have
the problem either. Back when FF10 & 11 were new they worked with 2.13.0.  Updating
Selenium/WebDriver is what broke it.  Even today FF 25.0.1 doesn't have this problem
when not under WebDriver.

Reported by bap0679 on 2013-11-20 21:52:28

[lukeis]

Extension method which accepts this notification and allows to continue

public static void HandleDocumentExpired(this RemoteWebDriver driver)
{
    try
    {
        IWebElement error = driver.FindElementById("errorTryAgain");
        error.Click();
        IAlert alert = driver.SwitchTo().Alert();
        alert.Accept();
    }
    catch (NoSuchElementException)
    {
    }
}

Reported by todizzy on 2013-11-28 14:10:48

[lukeis]

Still facing this issue and Todi's code did not work for me

Reported by cyaskoski on 2014-03-03 16:43:50

[lukeis]

So I wrote a little method that first checked if element ID "errorTitle" exists. If
so it then checks for the text "Document Expired". If it finds that it then clicks
element ID "errorTryAgain". My tests now work and it is able to click try again on
the Document Expired page.

Reported by cyaskoski on 2014-03-03 19:17:17

[lukeis]

its same as Todi's code...

driver.findelement(By.id("errorTryAgain")).click();
Alert alt=driver.switchTo().alert();
alt.accept();


its working fine in my case...

Reported by RojeshMishra on 2014-05-13 09:55:02

[lukeis]

Reported by luke.semerau on 2015-09-17 17:44:49

• Labels added: Restrict-AddIssueComment-Commit