Conversation
| // Wait 1 second before retrying | ||
| sleep(1) |
There was a problem hiding this comment.
Is the sleep necessary when we already wait for waitTimeout?
There was a problem hiding this comment.
If the sleep is necessary, I'd suggest making it configurable.
There was a problem hiding this comment.
in this case, i think it's necessary for that short wait before retrying again and should be a really low number. i'll make it configurable
| } catch { | ||
| if currentRetryCount < maxRetries { | ||
| // Wait 1 second before retrying | ||
| sleep(1) | ||
| continue | ||
| } else { | ||
| throw WaitForScreenError.timedOut | ||
| } |
There was a problem hiding this comment.
I'm not sure if this is the correct syntax, but we should be able to forward the error thrown from the do try part of the code, without having to replicate it.
| } catch { | |
| if currentRetryCount < maxRetries { | |
| // Wait 1 second before retrying | |
| sleep(1) | |
| continue | |
| } else { | |
| throw WaitForScreenError.timedOut | |
| } | |
| } catch error { | |
| if currentRetryCount < maxRetries { | |
| // Wait 1 second before retrying | |
| sleep(1) | |
| continue | |
| } else { | |
| throw error | |
| } |
|
I've been thinking about how to test this in the UI tests for the library, but I can't come up with anything that would be reliable... 🤔 The best I can think of is a screen with a timer. I wonder if that would be robust or flaky. It'd be interesting if it was flaky because it would give us a way to make the library stronger. At the same time, it'd be annoying to have to deal with flaky tests here, too. |
Description
Some tests that have 100% reliability on iPhone are flaky on iPad. The top two are:
Found that on iPad, certain flows open up a modal on top of a screen instead of redirecting to a new screen. When this happens, sometimes, the test hangs when trying to find the right element on the screen. I suspect that when the test reaches the part of the code to verify for elements, the modal has not rendered correctly, so the test cannot find what it's looking for. This PR adds a retry to
waitForScreenso if, at that point, the screen has not been rendered, it can try again (max 3 retries).Note after testing: For testCreateScheduledPost, found that the reason that this is happening is that the notice disappears before the test has the chance to tap it, this issue will be tackled separately and not fixed by the changes in this PR
One more note: A surprising but pleasant bonus, I discovered that the issue where the first test
testInsightsStatsLoadProperlywould fail consistently on the first run on the simulator, is resolved in this PR. Please see all the test runs after wordpress-mobile/WordPress-iOS@dc9586c commit in the testing PRTesting
Using wordpress-mobile/WordPress-iOS#21213 to test the changes, tests ran multiple times and checked that this failure does not happen on iPad for
testWPcomLoginLogouttest: