Fix flaky iPad UI Test - testWPcomLoginLogout#21200
Conversation
|
| App Name | WordPress Alpha |
|
| Configuration | Release-Alpha | |
| Build Number | pr21200-7bc3089 | |
| Version | 22.9 | |
| Bundle ID | org.wordpress.alpha | |
| Commit | 7bc3089 | |
| App Center Build | WPiOS - One-Offs #6524 |
|
| App Name | Jetpack Alpha |
|
| Configuration | Release-Alpha | |
| Build Number | pr21200-7bc3089 | |
| Version | 22.9 | |
| Bundle ID | com.jetpack.alpha | |
| Commit | 7bc3089 | |
| App Center Build | jetpack-installable-builds #5555 |
pachlava
left a comment
There was a problem hiding this comment.
@jostnes 👋
Thanks for working on this! While in theory the change should not reduce the flakiness, because the change is about making MeTabScreen be instantiated separately, instead of having it as a return value of goToMeScreen 🤔 (correct me please, if I'm missing something), let give it a go and see how it runs. The CI is green, so ![]()
P.S. While launching the Simulator locally for the first time, this was blocking the execution:
It looks like this does not happen on CI, though.
|
@pachlava thank you for reviewing!
yes that is correct, doesn't really make sense in theory (because it should already work as is! 😅) so a bit of experimenting to see if separating it would help with the flakiness on iPad
that looks new, never seen it. will monitor and see if we need to handle that soon... |



Description
The
testWPcomLoginLogouttest is flaky on iPad test runs. On iPhone test runs, it has a 100% passing rate.When looking at the difference between iPad and iPhone, I found that on iPad, the
MeTabScreenis a modal on top of theMySiteScreen. Because there is often a bit of a timing issue in CI, when the test is verifying thatMeTabScreenis loaded, it's still testing againstMySiteScreenelement tree causing the test to hang and fail.I'm not 100% sure this will be the right solution because, in theory, the test without any changes should work, but that isn't the reality. I'd like to try this out and monitor for a bit if it works. There are also other iPad-only flakiness that could be because of the same issue. If this works, we can apply the same to those tests too.
Testing
testWPcomLoginLogoutis not flaky on iPad (with error:Failed to determine hittability of "appSettings" Cell: Activation point invalid and no suggested hit points based on element frame) and CI is 🟢