Skip to content

Fix flaky verifyOnAppComesFromBackgroundCalled test#17370

Merged
irfano merged 1 commit intotrunkfrom
fix/flaky-verifyonappcomesfrombackgroundcalled-test
Oct 28, 2022
Merged

Fix flaky verifyOnAppComesFromBackgroundCalled test#17370
irfano merged 1 commit intotrunkfrom
fix/flaky-verifyonappcomesfrombackgroundcalled-test

Conversation

@irfano
Copy link
Copy Markdown
Member

@irfano irfano commented Oct 25, 2022

This increases sleep time in verifyOnAppComesFromBackgroundCalled test case.

verifyOnAppComesFromBackgroundCalled fails rarely on Firebase Test Lab environment.
I ran the application multiple times on emulator and debugged the order of related functions. It's like this:

  1. App is launched,
  2. Application's onCreate runs,
  3. AppInitializer's onAppComesFromBackground runs,
  4. WordPress.appIsInTheBackground is set to false.
  5. If we run verifyOnAppComesFromBackgroundCalled test case, the test runs in this step.

But in FTL environment, step 4 works before 2 sometimes. That causes the test to fail. We don't use WordPress.appIsInTheBackground for instant checks after app launch, so it's totally safe to increase sleep time for verifyOnAppComesFromBackgroundCalled. The time was set to 200, but it can be increased if it still fails.

To test:
Run verifyOnAppComesFromBackgroundCalled test case on FTL a couple times.

Regression Notes

  1. Potential unintended areas of impact
    None

  2. What I did to test those areas of impact (or what existing automated tests I relied on)
    N/A

  3. What automated tests I added (or what prevented me from doing so)
    Updated current test.

PR submission checklist:

  • I have completed the Regression Notes.
  • I have considered adding accessibility improvements for my changes.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@irfano irfano added this to the 21.1 milestone Oct 25, 2022
@irfano irfano requested a review from pachlava October 25, 2022 20:12
@irfano irfano requested a review from a team as a code owner October 25, 2022 20:12
@wpmobilebot
Copy link
Copy Markdown
Contributor

Jetpack📲 You can test these changes on Jetpack by downloading jetpack-installable-build-pr17370-81120a0.apk
💡 Scan this QR code with your Android phone to download and install the APK directly on it.
AppJetpack
Build FlavorJalapeno
Build TypeDebug
Commit81120a0
Note: This installable build uses the JalapenoDebug build flavor, and does not support Google Login.

@wpmobilebot
Copy link
Copy Markdown
Contributor

WordPress📲 You can test these changes on WordPress by downloading wordpress-installable-build-pr17370-81120a0.apk
💡 Scan this QR code with your Android phone to download and install the APK directly on it.
AppWordPress
Build FlavorJalapeno
Build TypeDebug
Commit81120a0
Note: This installable build uses the JalapenoDebug build flavor, and does not support Google Login.

Copy link
Copy Markdown
Contributor

@pachlava pachlava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for working on this @irfano 👋

I can see that Instrumented Tests are green on FTL, and I think it's enough to have the change merged. Because the test rarely fails, the best way is to let the change into the wild and see how it behaves in the long run 👍 🚀

@irfano irfano merged commit 652093f into trunk Oct 28, 2022
@irfano irfano deleted the fix/flaky-verifyonappcomesfrombackgroundcalled-test branch October 28, 2022 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants