Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

App uninstalls and reinstalls for every test file when using Jest #1279

Closed
1 task done
META-DREAMER opened this issue Apr 15, 2019 · 11 comments
Closed
1 task done

App uninstalls and reinstalls for every test file when using Jest #1279

META-DREAMER opened this issue Apr 15, 2019 · 11 comments

Comments

@META-DREAMER
Copy link
Contributor

Describe the bug
After refactoring our tests to use Jest instead of Mocha, Detox started uninstalling / reinstalling the app for every test file. This greatly slows down our tests and is not ideal.

To Reproduce

  • I have tested this issue on the latest Detox release and it still reproduces

Any Detox setup with Jest will exhibit this behaviour while Mocha setups will not.

Expected behavior
The app should not be reinstalled for every test file unless the user tells detox to do so.

Environment (please complete the following information):

  • Detox: 12.2.1
  • Jest: 24.4.0
  • React Native: 0.58.6
  • Node: 8.11.2
  • Device: iPhone X
  • Xcode: 10
  • iOS: 12.1
  • macOS: Mojave

Device and Verbose Detox Logs
Example of it reinstalling between tests

detox[14058] DEBUG: [exec.js/EXEC_CMD, #11] /usr/bin/xcrun simctl shutdown FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_TRY, #11] Shutting down FCC3C45C-8F27-44F6-8E0B-AB5A271D1532...
detox[14058] DEBUG: [DetoxServer.js/DISCONNECT] role=tester, sessionId=386a6d9f-0821-fab3-433c-e7e7d1428729
detox[14058] DEBUG: [DetoxServer.js/DISCONNECT] role=testee, sessionId=386a6d9f-0821-fab3-433c-e7e7d1428729
PASS e2e/tests/Contacts.spec.ts (380.165s)
  Checkins
    ✓ User adds a new contact to the space via checkin screen|user can add contact note on new checkin (53889ms)
    ✓ User adds a new contact via the select contact screen (49995ms)
    ✓ user performs checkin with existing contact|user adds inspection note on checkin|user edits contact while checking it in (84214ms)
  Contacts
    ✓ user can search for existing contact by name and mobile (26458ms)
    ✓ Create new contact and check contact is created (65661ms)
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #11] FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 shut down
detox[14058] INFO:  [DetoxServer.js] server listening on localhost:50929...
detox[14058] DEBUG: [AsyncWebSocket.js/WEBSOCKET_OPEN] opened web socket to: ws://localhost:50929
detox[14058] DEBUG: [DetoxServer.js/LOGIN] role=tester, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=tester, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [exec.js/EXEC_CMD, #0] /usr/bin/xcrun simctl list -j
detox[14058] DEBUG: [exec.js/EXEC_CMD, #1] applesimutils --list --byType "iPhone X" --byOS "12.2"
detox[14058] DEBUG: [exec.js/EXEC_TRY, #1] Searching for device matching iPhone X...
detox[14058] DEBUG: [exec.js/EXEC_CMD, #2] applesimutils --list --byId "FCC3C45C-8F27-44F6-8E0B-AB5A271D1532"
detox[14058] DEBUG: [exec.js/EXEC_CMD, #3] xcodebuild -version
detox[14058] DEBUG: [exec.js/EXEC_CMD, #4] /usr/bin/xcrun simctl boot FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_TRY, #4] Booting device FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_CMD, #5] /usr/bin/xcrun simctl bootstatus FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_CMD, #6] /usr/bin/xcrun simctl uninstall FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer
detox[14058] DEBUG: [exec.js/EXEC_TRY, #6] Uninstalling com.homepass.consumer...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #6] com.homepass.consumer uninstalled
detox[14058] DEBUG: [exec.js/EXEC_CMD, #7] /usr/bin/xcrun simctl install FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 "/Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app"
detox[14058] DEBUG: [exec.js/EXEC_TRY, #7] Installing /Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #7] /Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app installed
detox[14058] DEBUG: [exec.js/EXEC_CMD, #8] /usr/bin/xcrun simctl terminate FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer
detox[14058] DEBUG: [exec.js/EXEC_TRY, #8] Terminating com.homepass.consumer...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #8] com.homepass.consumer terminated
detox[14058] DEBUG: [exec.js/EXEC_CMD, #9] applesimutils --simulator FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 --bundle com.homepass.consumer --setPermissions notifications=YES,location=always
detox[14058] DEBUG: [exec.js/EXEC_TRY, #9] Trying to set permissions...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #9] Permissions are set
detox[14058] DEBUG: [exec.js/EXEC_CMD, #10] /bin/cat /dev/null >/Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.out 2>/Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/vagrant/Library/Detox/ios/7f772a17b6b8438f781c4abce46983bf49cbca74/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer --args -detoxServer ws://localhost:50929 -detoxSessionId cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [exec.js/EXEC_TRY, #10] Launching com.homepass.consumer...
detox[14058] INFO:  [AppleSimUtils.js] com.homepass.consumer launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F /Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.{out,err}
detox[14058] DEBUG: [DetoxServer.js/CANNOT_FORWARD] role=testee not connected, cannot fw action (sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d)
detox[14058] DEBUG: [DetoxServer.js/LOGIN] role=testee, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=testee, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
@noomorph
Copy link
Collaborator

noomorph commented Apr 15, 2019

Just a wild suggestion - do you run with --runInBand?

At the moment I'm not sure how to do that exactly, but in your shoes, I'd try my luck with https://jestjs.io/docs/en/configuration#globalsetup-string, do you agree?

Is there a blocker from Detox side preventing using globalSetup?

@META-DREAMER
Copy link
Contributor Author

@noomorph Problem with --runInBand is that detox automatically adds the --maxWorkers 1 option when calling jest. Jest doesn't allow specifying both runInBand and maxWorkers.

How exactly would globalSetup help with the issue? What would I be doing in the globalSetup?

@noomorph
Copy link
Collaborator

@HammadJ , I suggest to reinstall the app on the emulator/simulator in a global setup. To stop reinstalling on each suite run, make sure to follow this:

https://github.com/wix/Detox/blob/master/docs/APIRef.Configuration.md#faster-test-runs-with-app-reuse

@META-DREAMER
Copy link
Contributor Author

@noomorph

I tried this by adding the following to my globalSetup.js:

const detox = require('detox');
const config = require('../package.json').detox;

module.exports = async () => {
    await detox.init(config, { launchApp: false });
    await detox.device.launchApp({
        newInstance: true,
        permissions: { notifications: 'YES', location: 'always' },
    });
}

And removing the init and launchApp from the beforeAll hook in the init.js file. However, this caused every single test to fail with errors like this:
image

I believe this is because the globalSetup is executed in a completely different context than the actual tests so the actual tests don't have a properly initialized detox.

@noomorph
Copy link
Collaborator

@HammadJ , I am still not sure if the guys from Detox team will approve my pull request to an example project, but that's basically it: #1328

WARNING: the approach you want is supported only in -w 1 (1 worker aka --runInBand) mode.

@support
Copy link

support bot commented Apr 23, 2019

We use the issue tracker exclusively for bug reports and feature requests. This issue appears to be a general usage or support question. Instead, please ask a question on Stack Overflow with the detox tag.

@support support bot closed this as completed Apr 23, 2019
Issues Triage automation moved this from Low priority to Closed Apr 23, 2019
@rotemmiz rotemmiz reopened this Apr 23, 2019
Issues Triage automation moved this from Closed to Needs triage Apr 23, 2019
@rotemmiz
Copy link
Member

I am reopening the issue just to explain what I know about this behaviour.

The nature of Jest as a multiworker test runner is causing this behaviour.

The test runner is spinning up workers, and dispatching jobs (in that case, test files) to each of them. AFAIK, there is no way to configure an env setup file per worker, only per test file.

--runInBand is exactly to --maxWorkers=1 (https://github.com/facebook/jest/blob/8e246a4f60107cd9c6738b5f48d158ee634fa82c/packages/jest-config/src/getMaxWorkers.ts#L15), so don't expect any different behaviour.

@noomorph, your suggested idea is interesting, but again, there's no lifecycle hook for per-worker startup AFAIK, therefore any setup solution with Jest will be limited.

@noomorph
Copy link
Collaborator

@HammadJ, could you please create a feature request for support of single reinstall in multi worker mode? We'll continue the discussion there.

@META-DREAMER
Copy link
Contributor Author

@noomorph Created a new issue here: #1331

Thanks for your time and effort on this!

@stale
Copy link

stale bot commented May 24, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.
Thank you for your contributions.

@stale stale bot added the 🏚 stale label May 24, 2019
@stale
Copy link

stale bot commented May 31, 2019

The issue has been closed for inactivity.

@stale stale bot closed this as completed May 31, 2019
Issues Triage automation moved this from Needs triage to Closed May 31, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Issues Triage
  
Closed
Development

No branches or pull requests

3 participants