Log-in step occasionally fails.

[stephenharris]

The "Given I am logged in as an " test scenario occasionally fails on Travis-CI with a "The user could not be logged-in" error.

Expected behavior

That the test passes all the time.

Current behavior

The test fails occasionally. No pattern has been observed yet.

Known failing instances:

Link	WP	PHP	Driver
https://travis-ci.org/stephenharris/behat-wordpress-extension/jobs/202793337	4.7.2	5.6	wpapi
https://travis-ci.org/paulgibbs/behat-wordpress-extension/jobs/205790372	4.7.2	5.6	wpcli
https://travis-ci.org/paulgibbs/behat-wordpress-extension/jobs/205790376	4.7.2	7.1	wpcli
https://travis-ci.org/paulgibbs/behat-wordpress-extension/jobs/206945238	4.7.2	7.1	wpcli

Steps to reproduce (for bugs)

The failure happens occasionally. Let's use the above table to identify instances.

[stephenharris]

This isn't new: stephenharris/WordPressBehatExtension#10

The problem appears to be the same. I can't find the screenshot of an instance of this bug, but it too showed the password entered into the username field.

[stephenharris]

Still happening occasionally. Here's a screenshot from my fork:

[paulgibbs]

This is pretty crazy.

[ghost]

Issue still exists for me when running tests locally or using Circle.CI.

[ghost]

It appears that I am getting log in error:

<div id="login_error">	<strong>ERROR</strong>: Invalid username. <a href="http://guwordpress.docksal/wp-login.php?action=lostpassword">Lost your password?</a><br></div>

[ghost]

There is definitely something wrong going on with setValue() method in logIn(). I called getValue() right after setValue() and it appears that the value is set improperly: a username is corrupted: in instead of myadmin, and empty string instead of a password.

[paulgibbs]

I think there’s some bug in Selenium that causes this. There are reports of similar issues with Mink and Selenium across the web, but they’re infrequent and very hard to find. That whole stack is very complex and it looks so hard to debug I haven’t tried. I think it’s safe to say until someone comes up with a reliable workaround or a fix, this is going to persist. The nearby calls to setFocus() were an attempt at this. My lead theory is because the log-in step is usually the first thing done in a new web browser session, if Selenium sometimes tries to send commands to the browser before it is ready, it might cause this sort of failure. As I write this, I wonder about logIn() — if it fails, try to log-in twice, as a workaround. I wonder if it might have any impact. It would prove whether this sort of failure is unique to the *first* command in a session or not. (Because the second attempt would be the second command)

…

--- Paul Gibbs

On 7 Feb 2018, at 19:21, Vadym Myrgorod ***@***.***> wrote: There is definitely something wrong going on with setValue() method in logIn(). I called getValue() right after setValue() and it appears that the value is set improperly: a username is corrupted: in instead of myadmin, and empty string instead of a password. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[ghost]

Adding extra setValue() fixed the problem, but this issue seems really weird!

Here is my patch:

diff -ur behat-wordpress-extension/src/Context/Traits/UserAwareContextTrait.php behat-wordpress-extension-patched/src/Context/Traits/UserAwareContextTrait.php
--- behat-wordpress-extension/src/Context/Traits/UserAwareContextTrait.php      2018-02-07 18:27:56.000000000 -0500
+++ behat-wordpress-extension-patched/src/Context/Traits/UserAwareContextTrait.php      2018-02-07 18:27:48.000000000 -0500
@@ -39,6 +39,7 @@
         }
         $node->setValue('');
         $node->setValue($username);
+        $node->setValue($username);

         $node = $page->findField('user_pass');
         try {
@@ -48,6 +49,7 @@
         }
         $node->setValue('');
         $node->setValue($password);
+        $node->setValue($password);

         $page->findButton('wp-submit')->click();

[ghost]

@paulgibbs just saw your comment. I tried to alter all tests with a background condition to load homepage first, but that didn't affect on the log in errors. Hope it helps.

[paulgibbs]

I think we tried duplicating the setValue calls in login() before and it did not permanently fix issue, so I’m skeptical. If you want to PR your change, the CI will build it and I can then run that multiple times to check how reliability it runs over multiple runs. Otherwise, I’ll make the PR, but this won’t be until next week.

…

--- Paul Gibbs

On 8 Feb 2018, at 03:57, Vadym Myrgorod ***@***.***> wrote: @paulgibbs just saw your comment. I tried to alter all tests with a background condition to load homepage first, but that didn't affect on the log in errors. Hope it helps. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[paulgibbs]

I've decided to merge it. I couldn't break it on Travis-CI, and I don't think it will hurt. I suspect whatever the very first step is, it might be occasionally prone to similar glitching, but without understanding the root cause more somehow, I've no better idea at the moment. :)

[paulgibbs]

Unsurprisingly, the latest commit here does not fix this problem. It lives on!

[paulgibbs]

Try https://stackoverflow.com/questions/26239561/behat-mink-selenium2-element-is-not-visible#26242599 next.

[ghost]

@paulgibbs it looks like this issue still exists. It does not happen quite often, but for a large amount of tests, it happens nearly in 30% of runs of test suites.

[paulgibbs]

Thanks for letting me know. What version are you testing with (or, what’s the the most recent version that causes this behaviour)?

…

--- Paul Gibbs

On 11 Jun 2018, at 23:22, Vadym Myrgorod ***@***.***> wrote: @paulgibbs it looks like this issue is still exists. It does not happen quite often, but for a large amount of tests, it happens nearly in 30% of builds. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[paulgibbs]

Latest attempted fix (with JS) and more information about the potential bug in Mink is detailed in #230.

[paulgibbs]

Behat/MinkExtension#286

[paulgibbs]

Going to close this for now -- #247 is the latest stab. Will re-open if it is still not more reliable etc.