iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3574

zacharyweidenbach · 2022-08-30T20:13:43Z

What happened?

First reported with a comment on a separate issue: #3342 (comment)

Running tests for iOS with 3 workers using the detox cli --workers leads to intermittent test failures. It seems to have something to do with the instantiation of new simulators to support the concurrent workers. When running only 1 worker, I am unable to reproduce this issue.

CI: Github Actions

As a workaround, I have created a simple test suite consisting of only 3 tests. These tests run with 3 workers so that Detox will create new simulator instances for them. This 3 tests usually fail with the timeout issue, however, when the actual test suite runs immediately after, it does not encounter the timeout issue. This leads me to believe that if the simulators have already been created, Detox bypasses this step, and bypasses the bug altogether.

Attached are the logs with trace logging enabled from when the errors occurred.

The command to run this step is as follows

 - name: Initialize Detox Tests
        id: detox-init
        continue-on-error: true
        run: yarn detox test --use-custom-logger false --loglevel warn --configuration ios.sim.gha --cleanup --workers 3 --runner-config e2e/InitCITests/config.json --keepLockFile true

What was the expected behaviour?

The jest-circus setup step would not timeout.

Was it tested on latest Detox?

I have tested this issue on the latest Detox release and it still reproduces.

Did your test throw out a timeout?

I have followed the instructions under Identifying which synchronization mechanism causes us to wait too much.

Help us reproduce this issue!

No response

In what environment did this happen?

Detox version: 19.10.0
React Native version: 0.67.2
Node version: 14.18.1
Device model: iPhone 13
iOS version: 14.5
macOS version: 12
Xcode version: 13.4.1
Test-runner (select one): jest-circus

Detox logs

18_Initialize Detox Tests-1.txt

Device logs

paste logs here!

More data, please!

No response

The text was updated successfully, but these errors were encountered:

gabrieldonadel · 2022-08-30T20:24:10Z

I've been experiencing this exact same problem for a while now, the only solution I found was to only use one worker at a time

stale · 2022-10-08T09:02:30Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.

stale · 2022-10-16T06:41:12Z

The issue has been closed for inactivity.

gabrieldonadel · 2022-10-16T14:34:22Z

Can we reopen this @d4vidi ?

AdamTyler · 2022-10-28T16:41:34Z

I'm having this issue on GH Actions after updating the runner from macos-11 to macos-12.

It seems the simulator doesn't boot fast enough since its the first use of it. I have maxWorkers at 1.
bootstatus is run but doesn't seem to report anything and eventually the timeout is fired and the tests don't get assigned to a simulator:

Oct 28 15:53:59 detox[45848] DEBUG: [EXEC_CMD, #2] /usr/bin/xcrun simctl boot 0336474D-27B1-4220-8EC9-DD82EDEBC404 
Oct 28 15:53:59 detox[45848] DEBUG: [EXEC_TRY, #2] Booting device 0336474D-27B1-4220-8EC9-DD82EDEBC404...
Oct 28 15:54:02 detox[45848] TRACE: [EXEC_SUCCESS, #2] 
Oct 28 15:54:02 detox[45848] DEBUG: [EXEC_CMD, #3] /usr/bin/xcrun simctl bootstatus 0336474D-27B1-4220-8EC9-DD82EDEBC404
Oct 28 15:55:58 detox[45848] ERROR: Exceeded timeout of 120000ms while handling jest-circus "setup" event
Oct 28 15:56:09 detox[45848] INFO:  Signup (Phone) is assigned to undefined

stale · 2022-11-27T21:42:44Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.

gabrieldonadel · 2022-11-27T23:51:13Z

This is still a problem

d4vidi · 2022-11-28T10:57:02Z

As far as I can tell from the logs, it appears that the simulators refuse to boot:

2022-08-30T17:24:37.6595290Z 17:24:35.471 detox[46305] TRACE: [EXEC_SUCCESS, #4] Monitoring boot status for iPhone 13-Detox (8B0919E4-BF8C-44C8-85C5-B327301D17F9).
2022-08-30T17:24:37.6698290Z [2022-08-30 17:17:04 +0000] Status=2, isTerminal=NO, Elapsed=03:18.
2022-08-30T17:24:37.6801620Z 	Waiting on Data Migration
2022-08-30T17:24:37.6904190Z 		Reason:Running plugin com.apple.managedconfiguration.migrator (MCProfile.migrator, user-agnostic)
2022-08-30T17:24:37.7011750Z 		Migration Elapsed:00:00 seconds
...
...
...
2022-08-30T17:24:46.4004910Z 		Reason:Running plugin com.apple.managedconfiguration.mdm.migrator (MCMDM.migrator)
2022-08-30T17:24:46.4105920Z 		Migration Elapsed:03:49 seconds
2022-08-30T17:24:46.4206390Z 
2022-08-30T17:24:46.4307930Z [2022-08-30 17:24:35 +0000] Status=2, isTerminal=NO, Elapsed=07:10.

Even after 7 minutes, they are stuck in the Data Migration step. Here's what a normal boot up looks like:

Monitoring boot status for iPhone 13-Detox (8F9D3538-64FA-4DF5-9492-E2C093E15748).
[2022-11-28 10:36:46 +0000] Status=2, isTerminal=NO, Elapsed=00:12.
	Waiting on Data Migration
		Reason:Running plugin com.apple.-0LaunchServicesMigrator (00LaunchServicesMigrator.migrator, user-agnostic)
		Migration Elapsed:00:10 seconds
...
...
...
[2022-11-28 10:36:52 +0000] Status=2, isTerminal=NO, Elapsed=00:17.
	Waiting on Data Migration
		Reason:Running plugin com.apple.managedconfiguration.mdm.migrator (MCMDM.migrator)
		Migration Elapsed:00:15 seconds

[2022-11-28 10:36:53 +0000] Status=2, isTerminal=NO, Elapsed=00:18.
	Waiting on Data Migration
		Reason:Running plugin com.apple.PreferencesMigrator (PreferencesMigrator.migrator)
		Migration Elapsed:00:16 seconds

[2022-11-28 10:36:54 +0000] Status=4, isTerminal=NO, Elapsed=00:19.
	Waiting on System App

[2022-11-28 10:36:55 +0000] Status=4, isTerminal=NO, Elapsed=00:21.
	Waiting on System App

[2022-11-28 10:36:56 +0000] Status=4, isTerminal=NO, Elapsed=00:21.
	Waiting on System App

[2022-11-28 10:36:57 +0000] Status=4, isTerminal=NO, Elapsed=00:22.
	Waiting on System App

[2022-11-28 10:36:58 +0000] Status=4, isTerminal=NO, Elapsed=00:23.
	Waiting on System App

[2022-11-28 10:36:58 +0000] Status=4, isTerminal=NO, Elapsed=00:24.
	Waiting on System App

[2022-11-28 10:36:59 +0000] Status=4, isTerminal=NO, Elapsed=00:25.
	Waiting on System App

[2022-11-28 10:37:01 +0000] Status=4, isTerminal=NO, Elapsed=00:26.
	Waiting on System App

[2022-11-28 10:37:02 +0000] Status=4, isTerminal=NO, Elapsed=00:27.
	Waiting on System App

[2022-11-28 10:37:03 +0000] Status=4294967295, isTerminal=YES, Elapsed=00:28.
	Finished

Something about the Github-actions environment is keeping Detox from creating the simulator dups it needs for concurrency. I don't have a solution for this. I can only propose some words of "wisdom":

Try to dig into Github-action's documentation and look for related limitations. You might want to consider running those commands Detox does in order to play with the environment by yourself, in order to come up with a workaround.
Consider creating simulator clones, the way Detox does, in advance. Detox will use those, if available.
Consider splitting your suite to several, complementary, Github actions.

remcoabalain · 2022-12-02T08:13:30Z

I've got the same problem with Detox in Microsoft Devops Pipelines, with one worker it works, with more than one, it fails. Same timeout result.

AdamTyler · 2022-12-16T19:25:57Z

@d4vidi I'm actually running into this issue without concurrency.
I'm trying to update my runner to macos-12 and iPhone 13 where previously I was on macos-11 and iPhone 11.

So I'm thinking there is some conflict with newer GH runners and Detox

brycnguyen · 2022-12-23T00:02:41Z

I'm also running into this same issue and can only run off 1 worker

huszzsolt · 2023-01-06T14:26:15Z

Have the same issue running 2 iOS workers on Bitrise CI/CD

stale · 2023-03-14T11:59:33Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.

gabrieldonadel · 2023-03-14T13:20:20Z

This is still a problem

Zakyyy · 2023-04-06T09:45:46Z

any solution

stale · 2023-05-08T20:45:57Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.

stale · 2023-05-20T11:27:44Z

The issue has been closed for inactivity.

marinefourton · 2023-05-26T10:09:01Z

This is still a problem

sergtimosh · 2023-09-27T07:26:25Z

@zacharyweidenbach could you please eleborate on --runner-config e2e/InitCITests/config.json part?
What does this alternative configration contains?

zacharyweidenbach · 2023-09-27T15:44:36Z

@sergtimosh

It is just a pretty standard config file

{
  "testEnvironment": "../environment",
  "testRunner": "jest-circus/runner",
  "testTimeout": 100000,
  "testRegex": "\\.initci\\.ts$",
  "reporters": ["detox/runners/jest/streamlineReporter"],
  "verbose": true
}

The test regex was targeting 3 test files called worker1.initci.ts, worker2.initci.ts, and worker3.initci.ts.
Each of these have a bare bones test in them

import { device } from 'detox';

test('It should start worker 1', () => device.launchApp());

I am no longer working on the codebase where I had this problem, but I was never able to solve it.

sergtimosh · 2023-09-27T16:06:55Z

@zacharyweidenbach

Thanks for the reply! By "I was never able to solve it" you mean solve with some pretty solution apart from the one you've stated in the topic, or it haven't work at all?

zacharyweidenbach · 2023-09-27T16:13:49Z

@sergtimosh

Sorry, I mean I was never able to solve this problem without the workaround.

zacharyweidenbach added platform: ios status: triage type: bug 🐞 labels Aug 30, 2022

zacharyweidenbach mentioned this issue Aug 30, 2022

Detox fails unless Android emulator is running prior to test execution - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3342

Closed

4 tasks

d4vidi added the bugathon label Sep 1, 2022

d4vidi changed the title ~~iOS Detox tests fail in CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event~~ iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event Sep 1, 2022

stale bot added the 🏚 stale label Oct 8, 2022

stale bot closed this as completed Oct 16, 2022

asafkorem reopened this Oct 18, 2022

stale bot removed the 🏚 stale label Oct 18, 2022

stale bot added the 🏚 stale label Nov 27, 2022

stale bot removed the 🏚 stale label Nov 27, 2022

d4vidi assigned asafkorem Jan 11, 2023

stale bot added the 🏚 stale label Mar 14, 2023

stale bot removed the 🏚 stale label Mar 14, 2023

stale bot added the 🏚 stale label May 8, 2023

stale bot closed this as completed May 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3574

iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3574

zacharyweidenbach commented Aug 30, 2022

gabrieldonadel commented Aug 30, 2022

stale bot commented Oct 8, 2022

stale bot commented Oct 16, 2022

gabrieldonadel commented Oct 16, 2022

AdamTyler commented Oct 28, 2022

stale bot commented Nov 27, 2022

gabrieldonadel commented Nov 27, 2022

d4vidi commented Nov 28, 2022 •

edited

Loading

remcoabalain commented Dec 2, 2022

AdamTyler commented Dec 16, 2022

brycnguyen commented Dec 23, 2022

huszzsolt commented Jan 6, 2023 •

edited

Loading

stale bot commented Mar 14, 2023

gabrieldonadel commented Mar 14, 2023

Zakyyy commented Apr 6, 2023

stale bot commented May 8, 2023

stale bot commented May 20, 2023

marinefourton commented May 26, 2023

sergtimosh commented Sep 27, 2023

zacharyweidenbach commented Sep 27, 2023

sergtimosh commented Sep 27, 2023 •

edited

Loading

zacharyweidenbach commented Sep 27, 2023

iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3574

iOS Detox tests fail in Github actions CI with multiple workers - Exceeded timeout of 300000ms while handling jest-circus "setup" event #3574

Comments

zacharyweidenbach commented Aug 30, 2022

What happened?

What was the expected behaviour?

Was it tested on latest Detox?

Did your test throw out a timeout?

Help us reproduce this issue!

In what environment did this happen?

Detox logs

Device logs

More data, please!

gabrieldonadel commented Aug 30, 2022

stale bot commented Oct 8, 2022

stale bot commented Oct 16, 2022

gabrieldonadel commented Oct 16, 2022

AdamTyler commented Oct 28, 2022

stale bot commented Nov 27, 2022

gabrieldonadel commented Nov 27, 2022

d4vidi commented Nov 28, 2022 • edited Loading

remcoabalain commented Dec 2, 2022

AdamTyler commented Dec 16, 2022

brycnguyen commented Dec 23, 2022

huszzsolt commented Jan 6, 2023 • edited Loading

stale bot commented Mar 14, 2023

gabrieldonadel commented Mar 14, 2023

Zakyyy commented Apr 6, 2023

stale bot commented May 8, 2023

stale bot commented May 20, 2023

marinefourton commented May 26, 2023

sergtimosh commented Sep 27, 2023

zacharyweidenbach commented Sep 27, 2023

sergtimosh commented Sep 27, 2023 • edited Loading

zacharyweidenbach commented Sep 27, 2023

d4vidi commented Nov 28, 2022 •

edited

Loading

huszzsolt commented Jan 6, 2023 •

edited

Loading

sergtimosh commented Sep 27, 2023 •

edited

Loading