Driver left in inconsistent state after error: The alert disappeared before it could be closed

[rajatjindal]

Issue seen on Platform: CentOS 6.5/Firefox 42.0/Selenium 2.48.2

works fine on: Mac OS/Firefox 42.0/Selenium 2.48.2

Test scripts/html can be found here: https://gist.github.com/rajatjindal83/4331f7fc6d5d9b819c3d

Steps to reproduce:

1. Download the attached testscripts and html files
2. Update scripts to have right location of html files
3. Run AlertTestOnMac.java, observe it works fine and behaves as expected by dismissing unhandled alert.
4. Run AlertTestOnCentOS.java, observe that it throws unhandled alert exception but don't dismiss alert.
5. Also observe that executing any other command, we get 'unhandled_alert_exception"
6. Also observe that executing 'acceptAlert' on this session, we get no_such_alert exception

Please let me know if I can provide more information.

Thanks
Rajat Jindal

[titusfortner]

I agree that this behavior should ideally be consistent, but it is easily worked around as is, and I disagree with you on the desired behavior. Firefox is the only browser that closes the alert when throwing the exception (and apparently even that is platform specific).

The w3c spec for this does not explicitly define a behavior for this situation, but as I read "When a user prompt appears, it is the task of the subsequent command to handle it," it implies that closing the alert should not be automatic when an exception is thrown.

1. Avoid flow control through rescuing Exceptions
2. Your test *should know when an alert pops up and your code should explicitly accept/dismiss it when it does
3. If it is truly a non-deterministic test where you can't know ahead of time, you can check for the existence of the alert as the condition for sending a close command before proceeding with your tests.

Because this behavior isn't explicitly defined by JSON Wire Protocol, and we are moving to Marionette anyway (which also does not dismiss the alert), this is either 'not a bug' or 'won't-fix.'

[rajatjindal]

Hi,

Thanks for looking into it and providing your feedback.

Can you please clarify what will be the workaround for this issue? I tried handling alert by myself by overriding using capabilities but it does not work.

Like you can see in code, it throws 'so such alert' exception when trying to handle alert and 'unexpected alert exception' when trying to execute other commands.

This is turning out to be a big blocker for us. Any help will be highly appreciated.

[titusfortner]

You should close the alert yourself when the popup shows up.
I didn't realize the popup didn't show for 5 seconds. You do have to poll and rescue exceptions in that case.

I don't know if Java has a special condition for waiting for an alert, or the syntax for accepting an alert, but in Ruby:

wait = Wait.new(:timeout => 5, :ignore => Error::NoAlertPresentError)
alert = wait.until { driver.switch_to.alert }
alert.accept

[rajatjindal]

Thanks for reviewing it again. One more thing that i think you missed while reviewing is that this testcase works perfectly fine when executed on MAC OS, but fails everytime executed on CentOS 6.5

We have huge number of tests, and we run selenium on our cloud running CentOS 6.5, and this issue is causing lots of noise in our tests.

With your permission, can I please re-open the bug for your kind consideration?

[rajatjindal]

Another thing is that i am unable to close the alert once this condition is reached. Basically I can't run any command after this without failure.

Any command to handle alert results in no alert exception, and any other command results in unhandled alert exception.

[titusfortner]

I get it. You have to close the alert, but the alert has gone away so you can't so the test has to die. Like I mentioned in my email reply, try polling the alert to be present by rescuing the not-present exceptions, then explicitly close it. If that doesn't work, I'll reopen it and we can figure out if there is something else going on.

[rajatjindal]

We have to let the session die not just the test. One session i used fir a
test suite which has more than one test,

If you look at the test script, you will understand more clearly.

Look at AlertTestForCentOS.java

On Saturday, November 21, 2015, Titus notifications@github.com wrote:

I get it. You have to close the alert, but the alert has gone away so you
can't so the test has to die. Like I mentioned in my email reply, try
polling the alert to be present by rescuing the not-present exceptions,
then explicitly close it. If that doesn't work, I'll reopen it and we can
figure out if there is something else going on.

—
Reply to this email directly or view it on GitHub
#1297 (comment)
.

Rajat Jindal | Gurgaon | http://quicksilver1183.com |
rajatjindal83@gmail.com

[titusfortner]

I'm saying re-write your code so you are never in the condition when you need to access an alert that has gone away.

[rajatjindal]

what I am trying to convey here is that, no matter how I try to handle this, it gets into this situation very often, and once in this situation, no matter what command I run, it always fails until I kill my complete test suite.

and when I say we have huge number of tests, we have about 18k tests, and a testsuite can have about 1000 testcases (worstcase, average tests are about 300 per test suite) and when this error occur all the remaining tests in same test suites can't run anymore.

Writing code to poll for alert after each click will increase the execution time by days (not hours) for us.

Traditionally we have used WebDriverBackedSelenium approach by overriding javascript alert and confirmation methods but that has its own set of issues, and therefore we are trying to move to native way of handling alerts and confirmations.

I will request you to give this issue little time, and I am sure with your expertise, you can quickly figure out whats going on once you run test scripts.

Can I please re-open this bug?

Thanks
Rajat Jindal

[titusfortner]

Your requirements confuse me.
You need to check for an alert after every click, but sometimes alerts don't show up for 5 seconds after a click?
Did you move your 18,000 tests from Mac to CentOS, or did this code previously work on CentOS? If so, what is different now?

[rajatjindal]

We were using overrides of alert and confirmation method to handle alerts,
like we do in WebdriverbackedSelenium, but it had its own set of issues as
we have to inject javascript before every click

( to give you more context we migrated these tests from selenium RC to
webdriver)

Here are the steps we generally do as part of testing our webapp:

1. Click on add button on original window
2. This opens a popup, we fill up form on popup and click save
3. This sends a request to backend and if its successful, it close popup
   automatically
4. If it fails due to some server side validation, it throws an alert
5. If it fails due to client side validation, it updates error msg text on
   popup and in some cases throw alert

So we dont want to check for alert after every command but handle it using
unhandled alert exception.

If we have to reset alert after receiving this exception explicitly, i am
perfectly fine with it, but like i said it does not allow me to run any
command if it reaches this state, not even to handle the alert msg,

I am happy to chat with you in selenium irc channel if you still have
questions

On Saturday, November 21, 2015, Titus notifications@github.com wrote:

Your requirements confuse me.
You need to check for an alert after every click, but sometimes alerts
don't show up for 5 seconds after a click?
Did you move your 18,000 tests from Mac to CentOS, or did this code
previously work on CentOS? If so, what is different now?

—
Reply to this email directly or view it on GitHub
#1297 (comment)
.

Rajat Jindal | Gurgaon | http://quicksilver1183.com |
rajatjindal83@gmail.com

[titusfortner]

Good on you for doing the migration to WebDriver; having done that before I know it is no small feat.

The bottom line for this project, though, is that w3c standard says that you must deal with an alert after one opens. As such, there is no requirement on our part to ensure that your test can recover from the exception that comes from not doing what the spec requires.

That being said, I recognize in the scenario you described how the non-determinant nature of that process can make real-world testing of it challenging. I would approach these with a helper method that submits the form, checks for an alert and fails the test if found, and moves on if not. Another thing I've done is to restart the browser after any test that could result in an Alert issue. You can also try running with Chrome. I know these aren't ideal solutions, but I still don't think this is an actual bug.

I'll ping the other devs on IRC on Monday when more of them are around, in case they disagree with me. :)

[titusfortner]

I didn't realize this was a capability option that Firefox supports. You can set unexpectedAlertBehaviour to "accept", "dismiss" or "ignore". Default behavior is to accept, so if you set it to ignore, then you can rescue the exception and then explicitly send the command to accept/dismiss it, and you shouldn't lose your session. Let me know if that works.

[rajatjindal]

Thanks for checking this out Titus. Appreciate that.

However setting unexpectedAlertBehaviour to ignore does not fix the problem.

[titusfortner]

Are you saying Firefox is not recognizing unexpectedAlertBehaviour?
At the very least you should be having a different problem.

[rajatjindal]

I agree. and thats something I am trying to convey by mentioning again and again that this issue happens on Firefox but only on CentOS platform. Same piece of code works on MAC. :)

Let me know how can I help you (ofcourse to help me), to understand and debug this problem?

Thanks
Rajat Jindal

[rajatjindal]

Hi Titus

could you please re-open this issue for further consideration.

Thanks
Rajat Jindal

[titusfortner]

I don't use Java to run your tests. Can you give me the output from each of these tests with ignore turned off & on? Are you running on centos with xvfb? via Jenkins?
I'll see if I can verify this with Ruby before re-opening. That being said, even if I verify it, I'm probably not going to be able to fix it, and most of the developer focus with Firefox right now is in the upcoming Marionette implementation, so addressing this isn't going to be a priority for anyone even if the issue is re-opened.