[SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 #853

bartolkaruza · 2018-07-24T11:15:36Z

Description

After upgrading to Detox 8.0.0 from 7.3.3, detox is not able to find the same iOS simulator using the name of the simulator. After downgrading, it works again, with no changes to the simulator config in between.

Error: a BeforeAll hook errored, process exiting: e2e/support/index.js:48: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"

Output of xcrun simctl list

== Devices ==
-- iOS 11.2 --
    DETOX (4E4CECC2-AA47-437F-B9B6-EC5917751329) (Booted)
-- iOS 11.3 --
    iPhone 7 (621A1BEF-EEF1-4017-BE73-A46DDAC3A0DF) (Shutdown)
-- tvOS 11.3 --
-- watchOS 4.3 --
== Device Pairs ==

``` Detox config in package.json
"detox": {
    "test-runner": "cucumber",
    "configurations": {
      "iphone": {
        "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/xx.app",
        "build": "yarn ios-detox-build",
        "type": "ios.simulator",
        "name": "DETOX"
      },
      "android": {
        "binaryPath": "./android/app/build/outputs/apk/debug/app-debug.apk",
        "build": "yarn android-detox-build",
        "type": "android.emulator",
        "name": "Nexus_5X_API_27"
      }
    }
  }

Steps to Reproduce

Create a working detox configuration with version 7.3.3
Upgrade to 8.0.0
Run tests

Detox, Node, Device, Xcode and macOS Versions

Detox: 8.0.0
React Native: 0.55
Node: v8.11.3
Device: iOS 11.2 simulator
Xcode: 9.3
macOS: 10.13.4

Device and verbose Detox logs

$ yarn e2e-test e2e/features/
$ ENVFILE=.env.e2e export $(cat .env.e2e | xargs -I{} echo 'DETOX_{}') && ./node_modules/.bin/cucumber-js --require-module babel-core/register --require-module babel-preset-stage-2 -f node_modules/cucumber-pretty --artifacts-location ./artifacts --configuration iphone e2e/features/
Started mock server on PORT 9000
detox info 13:03:50: server listening on localhost:64969...
detox info 2: Searching for device matching DETOX... 
VError: a BeforeAll hook errored, process exiting: e2e/support/index.js:48: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at /Users/xx/dev/proj/x/mobile-app/node_modules/cucumber/src/runtime/index.js:36:15
    at Generator.next (<anonymous>)
    at Generator.tryCatcher (/Users/xx/dev/proj/x/mobile-app/node_modules/bluebird/js/release/util.js:16:23)
    at PromiseSpawn._promiseFulfilled (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/generators.js:97:49)
    at Promise._settlePromise (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:574:26)
    at Promise._settlePromise0 (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
caused by: Error: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at AppleSimUtils.findDevicesUDID (/Users/xx/dev/proj/xx/mobile-app/node_modules/detox/src/devices/AppleSimUtils.js:50:13)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
error Command failed with exit code 1.

The text was updated successfully, but these errors were encountered:

bartolkaruza · 2018-07-24T11:38:22Z

It appears the semantics of the name in configuration has changed. Where in 7.3.3 the name field corresponded to the AppleSimUtils --byName flag, it now only corresponds to the nested deviceType.name field, which is really a deviceType, not a name. After changing the name to iPhone 7 it was able to access a simulator, but the wrong one.

The reason that it is very important to us to be able to select a specific simulator for the UI tests, is because the UI test simulator has configurations that are not ergonomic for normal development use simulator; eg no hardward keyboard.

output of applesimutils --list

[
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "iPhone 7",   <---- Development device
    "udid" : "621A1BEF-EEF1-4017-BE73-A46DDAC3A0DF",
    "os" : {
      "version" : "11.3",
      "availability" : "(available)",
      "name" : "iOS 11.3",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-3",
      "buildversion" : "15E217"
    }
  },
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "DETOX", <---- UI Test device
    "udid" : "4E4CECC2-AA47-437F-B9B6-EC5917751329",
    "os" : {
      "version" : "11.2",
      "availability" : "(available)",
      "name" : "iOS 11.2",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-2",
      "buildversion" : "15C107"
    }
  }
]

output of applesimutils --list --byName DETOX

[
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "DETOX",  <--- UI test device
    "udid" : "4E4CECC2-AA47-437F-B9B6-EC5917751329",
    "os" : {
      "version" : "11.2",
      "availability" : "(available)",
      "name" : "iOS 11.2",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-2",
      "buildversion" : "15C107"
    }
  }
]

If this was the intended behaviour before 8.0.0 and the functionality we rely on was a bug to begin with, can you let us know how we could specify a specific simulator using the configuration?

noomorph · 2018-07-25T05:48:08Z

It does look like a not documented breaking change, but not sure yet. I'll advise with @rotemmiz on this.

LeoNatan · 2018-07-25T09:43:49Z

This is not good and must be resolved. There is no reason to break this.

noomorph · 2018-07-25T09:47:26Z

@LeoNatan, I agree and I'll speak to Rotem at the first opportunity.

etaiklein · 2018-07-30T13:54:29Z

Related: https://stackoverflow.com/questions/51594729/detox-the-request-was-denied-by-service-delegate-pbprocessmanager-for-reason#51595404

chadlwilson · 2018-08-05T06:47:05Z

Yeah, this also broke our hand-rolled parallelization of Detox tests we are using on pre-8.0 versions. We are using xcrun to script creation of 'n' simulator devices with unique names (but same underlying device type and iOS version) and using gulp to split our tests to run against them, then using an env var to tell Detox which simulator instance to use for each batch.

Detox 8 seems to have broken this; I believe because the semantics of AppleSimUtils's search for devices by UDID has been completely changed to use a 'byType' flag, with an attempt to find the latest OS version for the given name: 94348fc#diff-2ec04c0d120ceb57a189afbc6a59a9e8

It feels to me that the new behaviour is designed around the assumption your Simulators are using "standard" names, like out-of-the-box simulators.

Unfortunately this change in semantics is preventing an incremental upgrade and conversion of our hand-rolled parallelization support to use Detox's inbuilt support :(

LeoNatan · 2018-08-05T09:40:54Z

Yes, this the issue. We discussed this internally and it's an issue we intend to fix. @rotemmiz @noomorph

There is an open issue from 8.0.0 and upward: wix/Detox#853 (comment)

DanielBoening · 2018-12-07T10:29:57Z

is it still planned to be fixed? The pull request seems to be ready but it is not yet merged.

vmaark · 2019-02-14T13:58:05Z

The same error happens when I try to run the react-native ios example. How can detox be used until the PR is merged? Older versions fail with the post install script.

stigi · 2019-05-21T10:41:35Z

Similar issue happened to me. I tried to create a dedicated simulator for my e2e test on CI using
xcrun simctl create "iPhone detox" com.apple.CoreSimulator.SimDeviceType.iPhone-X com.apple.CoreSimulator.SimRuntime.iOS-12-2

The entry in my package.json looks like this:

"ios.sim.release": {
  "binaryPath": "build/kollex/Build/Products/Release-iphonesimulator/kollex.app",
  "build": "xcodebuild -workspace ios/kollex.xcworkspace -scheme kollex -configuration Release -sdk iphonesimulator -derivedDataPath build/kollex",
  "type": "ios.simulator",
  "name": "iPhone detox"
}

But detox can't find the device:

detox[51803] ERROR: [index.js/DETOX_INIT_ERROR]
 Error: Can't find a simulator to match with "6E69D11C-6814-468F-92EC-F30A183EAC72", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at AppleSimUtils.findDevicesUDID (/Users/ullrich/Projects/kollex-app/node_modules/detox/src/devices/ios/AppleSimUtils.js:47:13)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)

even though applesimutils finds it:

~ applesimutils --list --byName "iPhone detox"
[
  {
    "availabilityError" : "",
    "os" : {
      "bundlePath" : "\/Applications\/Xcode.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Developer\/Library\/CoreSimulator\/Profiles\/Runtimes\/iOS.simruntime",
      "availabilityError" : "",
      "buildversion" : "16E226",
      "availability" : "(available)",
      "isAvailable" : true,
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-12-2",
      "version" : "12.2",
      "name" : "iOS 12.2"
    },
    "availability" : "(available)",
    "deviceType" : {
      "name" : "iPhone X",
      "bundlePath" : "\/Applications\/Xcode.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Developer\/Library\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone X.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-X"
    },
    "udid" : "6E69D11C-6814-468F-92EC-F30A183EAC72",
    "isAvailable" : true,
    "state" : "Booted",
    "name" : "iPhone detox"
  }
]

Env

detox: 12.8.0
applesimutils: 0.6.4

SEAPUNK · 2019-05-24T00:59:13Z

What's the workaround for this? We're trying to add E2E testing with Detox, but not being able to specify a device name is pretty much a blocker for us.

noomorph · 2019-05-25T05:09:25Z

@SEAPUNK, planned for June, as far as I am concerned.

jeromecornet · 2019-06-03T23:48:33Z

FYI I believe that the code in #1404 fixes this (although it raises question on the parameter name)

noomorph added the status: discussion label Jul 25, 2018

LeoNatan added type: bug 🐞 and removed status: discussion labels Jul 25, 2018

LeoNatan added the 📌 pinned label Aug 5, 2018

noomorph changed the title ~~Simulator not found after upgrading to 8.0.0~~ [SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 Aug 7, 2018

noomorph mentioned this issue Aug 29, 2018

[WIP] Returns back the previous device selection logic on iOS simulator #905

Closed

3 tasks

agazso added a commit to felfele/felfele that referenced this issue Oct 1, 2018

Downgraded detox to 7.3.3

ed0cb59

There is an open issue from 8.0.0 and upward: wix/Detox#853 (comment)

agazso mentioned this issue Oct 1, 2018

Detox felfele/felfele#5

Merged

noomorph mentioned this issue Apr 2, 2019

Incorrect OS version of specified iPhone simulator name #1103

Closed

noomorph added this to Accepted in Issues Triage Apr 18, 2019

jeromecornet mentioned this issue Jun 2, 2019

Allow specifying the simulator name or UDID as a device name for iOS simulators #1404

Closed

1 task

This was referenced Oct 8, 2019

refactor: device selection revisited #1694

Merged

feat(configuration): name string -> device object #1698

Merged

noomorph closed this as completed in #1698 Oct 15, 2019

Issues Triage automation moved this from Accepted to Closed Oct 15, 2019

lock bot locked as resolved and limited conversation to collaborators Oct 18, 2019

d4vidi added the status: accepted label Apr 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 #853

[SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 #853

bartolkaruza commented Jul 24, 2018 •

edited

bartolkaruza commented Jul 24, 2018

noomorph commented Jul 25, 2018 •

edited

LeoNatan commented Jul 25, 2018

noomorph commented Jul 25, 2018

etaiklein commented Jul 30, 2018

chadlwilson commented Aug 5, 2018 •

edited

LeoNatan commented Aug 5, 2018

DanielBoening commented Dec 7, 2018

vmaark commented Feb 14, 2019

stigi commented May 21, 2019

SEAPUNK commented May 24, 2019 •

edited

noomorph commented May 25, 2019

jeromecornet commented Jun 3, 2019

[SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 #853

[SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0 #853

Comments

bartolkaruza commented Jul 24, 2018 • edited

Description

Steps to Reproduce

Detox, Node, Device, Xcode and macOS Versions

Device and verbose Detox logs

bartolkaruza commented Jul 24, 2018

noomorph commented Jul 25, 2018 • edited

LeoNatan commented Jul 25, 2018

noomorph commented Jul 25, 2018

etaiklein commented Jul 30, 2018

chadlwilson commented Aug 5, 2018 • edited

LeoNatan commented Aug 5, 2018

DanielBoening commented Dec 7, 2018

vmaark commented Feb 14, 2019

stigi commented May 21, 2019

Env

SEAPUNK commented May 24, 2019 • edited

noomorph commented May 25, 2019

jeromecornet commented Jun 3, 2019

bartolkaruza commented Jul 24, 2018 •

edited

noomorph commented Jul 25, 2018 •

edited

chadlwilson commented Aug 5, 2018 •

edited

SEAPUNK commented May 24, 2019 •

edited