Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: stable iOS e2e tests #392

Merged
merged 57 commits into from
Apr 2, 2024
Merged

fix: stable iOS e2e tests #392

merged 57 commits into from
Apr 2, 2024

Conversation

kirillzyusko
Copy link
Owner

@kirillzyusko kirillzyusko commented Mar 19, 2024

📜 Description

Make iOS tests on CI stable again.

💡 Motivation and Context

Looks like after updating some of runners to XCode 15.3 booting up a simulator takes long time (actually it boots pretty fast, but after that CPU usage is close to 100% for quite big period of time). As a result tests are failing because of timeout.

I've tried many things - switching to other XCode versions, increasing timeout, changing the order of tests, changing runners, pre-booting simultator etc. and still tests very flaky in some cases.

However what I discovered is that after ~20mins and after several failures tests start to work okay. So I decided to add --retry 2 flag to overcome the problem of flaky simulator - it makes CI green again. But if a regression really introduced - it'll fail anyway just with linger time of execution 🙂

📢 Changelog

E2E

  • added patch-package;
  • updated detox;
  • updated assets;
  • moved tests preparation in jest-setup.ts file;
  • run tests with retries;
  • use scroll instead of swipe in AwareScrollView test;

CI

  • increased timeout (60 -> 90);

🤔 How Has This Been Tested?

Tested on CI.

📝 Checklist

  • CI successfully passed
  • I added new mocks and corresponding unit-tests if library API was changed

@kirillzyusko kirillzyusko added 🍎 iOS iOS specific CI CI (github actions) e2e Anything about E2E tests labels Mar 19, 2024
@kirillzyusko kirillzyusko self-assigned this Mar 19, 2024
Copy link
Contributor

github-actions bot commented Mar 19, 2024

📊 Package size report

Current size Target Size Difference
130819 bytes 130819 bytes 0 bytes 📉

@kirillzyusko
Copy link
Owner Author

Looks like:

  • switching to XCode 15.2, 14.3 didn't help
  • installation of rosetta didn't help
  • using latest version of detox didn't help
  • using iOS 16.4 didn't help

I opened an issue in detox - wix/Detox#4415

@kirillzyusko kirillzyusko added the deps update Upgrade dependencies of the project label Mar 27, 2024
@kirillzyusko kirillzyusko force-pushed the fix/stable-ios-e2e branch 2 times, most recently from 758ebe0 to 7eb6754 Compare March 27, 2024 17:07
@kirillzyusko kirillzyusko marked this pull request as ready for review April 2, 2024 13:03
@kirillzyusko kirillzyusko merged commit fdd7f84 into main Apr 2, 2024
7 checks passed
@kirillzyusko kirillzyusko deleted the fix/stable-ios-e2e branch April 2, 2024 13:11
@kirillzyusko kirillzyusko mentioned this pull request May 7, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI CI (github actions) deps update Upgrade dependencies of the project e2e Anything about E2E tests 🍎 iOS iOS specific
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant