[Tests] Page refresh on retry logic causing tests to fail

[stacey-gammon]

When I run the selenium tests locally I sometimes see flashes of the fatal splash screen, followed by a page refresh. Most often the test would continue without failing, but there is one spot where after the fatal splash screen, it won't recover.

That spot is here in dashboard_page.js, the saveDashboard function:

      // verify that green message at the top of the page.
      // it's only there for about 5 seconds
      await retry.try(() => {
        log.debug('verify toast-message for saved dashboard');
        return getRemote()
          .findByCssSelector('kbn-truncated.toast-message.ng-isolate-scope')
          .getVisibleText();
      });

I suspect the issue is that if the retry logic catches an error, it refreshes the page which causes the toast message to go away, so then it never recovers.

@archanid - do you know if there was any change to the retry logic? I don't recall seeing the fatal notification screen, even when tests passed, prior to the big test refactor. I think this may be an issue for spots that are testing temporary state.

I'm wondering if it's related to #11339

[stacey-gammon]

I haven't seen this in awhile so closing.