Unable to start UI Test on ios simulator

[jpc1963]

What App Center service does this affect?
iOS Simulators

Describe the bug
I have a Xamarin UI test project with tests apps on iOS and Android simulators.
Recently I had to update xCode (to 10.2) and since the update I can no longer start the iOS app on the simulator from my UI Test project. I just get the error TestFixtureSetUp failed in "myTestProject”.

To Reproduce
The app is started as follows:

public override IApp StartApp(AppDataMode ClearAppData)
{
  return ConfigureApp
   .iOS
   .InstalledApp("myAppNAme")
   .DeviceIdentifier(123456-123456) 
   .PreferIdeSettings()
   .EnableLocalScreenshots()
   .WaitTimes(new WaitTimes())
   .Debug()
   .StartApp(ClearAppData);
}

And this command returns the following error/exception: Unable to determine simulator version for 123456-123456

Expected behavior
I expect the iOS simulator to be started. The simulator ID is good. I can see it listed when I list my iOS simulators. And it says that the simulator is Booted. This all worked fine before the xCode update.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: MacOS 10.14.4
• Xamarin: 7.8.3

[galfonso777]

Having this same issue. Able to launch the simulator manually with no issues in VS for Mac but when launching a UITest VS displays the following:
SetUp : System.Exception : Unable to determine simulator version for 'X'

Visual Studio Enterprise 2019 for Mac
Version 8.0 (build 3001)
Xamarin.iOS
Version: 12.6.0.25 (Visual Studio Enterprise)
Operating System
Mac OS X 10.14.4
Xcode
Version 10.2 (10E125)

[sklmnv]

Same issue. I have performed a clean installation of the entire local development environment:
Visual Studio: v8.0 (build 3001)
xcode 10.2

App is created using default xamarin forms template.

[zanycadence]

We're experiencing the same issue. Attempting to run UITest locally results in the 'Unable to determine simulator version...' error for all iOS tests.

[patniko]

Hi @jpc1963, I am assuming this is an issue related to Visual Studio for Mac? We're in the Visual Studio family, but App Center is a different service that just so happens to support Xamarin. :-)

@pierceboggan can you help?

[zanycadence]

Yes, running Xamarin.UITest locally on a Mac, testing iOS builds. All of the MS threads on this issue say to contact appcenter. No comments from anyone at MS on the github issue either. Any help would be appreciated. Thanks,

…

-Eli

On Fri, Apr 5, 2019, 4:45 PM Patrick Nikoletich ***@***.***> wrote: Hi @jpc1963 <https://github.com/jpc1963>, I am assuming this is an issue related to Visual Studio for Mac? We're in the Visual Studio family, but App Center is a different service that just so happens to support Xamarin. :-) @pierceboggan <https://github.com/pierceboggan> can you help? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#274 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADDAnk8JscMpxUCtKZtXswdTZ5aKRfTRks5vd-ApgaJpZM4cZzco> .

[GeorgeDarakchiev]

Hi @patniko can you elaborate why are you assuming that this is related to Visual Studio for Mac?

Isn't this more case of an issue with Xcode 10.2 as suggested in this thread -> https://stackoverflow.com/questions/55567333/ui-test-not-able-to-run-ios-ui-tests-unable-to-determine-simulator-version-fo

Here is the stacktrace:

at Xamarin.UITest.iOS.Instruments.GetMajorVersionForSimulator (System.String deviceIdentifier) [0x0008c] in <2a16c16730a54859bda72c6bc1c728f7>:0 
  at Xamarin.UITest.iOS.iOSAppLauncher.LaunchAppLocal (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Http.HttpClient httpClient, System.Boolean clearAppData) [0x0011a] in <2a16c16730a54859bda72c6bc1c728f7>:0 
  at Xamarin.UITest.iOS.iOSAppLauncher.LaunchApp (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Http.HttpClient httpClient, Xamarin.UITest.TestCloud.TestCloudiOSAppConfiguration testCloudAppConfiguration, Xamarin.UITest.Shared.Http.HttpClient testCloudWsClient, Xamarin.UITest.Shared.Http.HttpClient xtcServicesClient, System.Boolean testCloudUseDeviceAgent) [0x00068] in <2a16c16730a54859bda72c6bc1c728f7>:0 
  at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration, Xamarin.UITest.Shared.Execution.IExecutor executor) [0x002af] in <2a16c16730a54859bda72c6bc1c728f7>:0 
  at Xamarin.UITest.iOS.iOSApp..ctor (Xamarin.UITest.Configuration.IiOSAppConfiguration appConfiguration) [0x00000] in <2a16c16730a54859bda72c6bc1c728f7>:0 
  at Xamarin.UITest.Configuration.iOSAppConfigurator.StartApp (Xamarin.UITest.Configuration.AppDataMode appDataMode) [0x00017] in <2a16c16730a54859bda72c6bc1c728f7>:0 

[maestersid]

@Oddj0b has Xamarin.UITest been updated to support XCode 10.2?

[agaluzzi]

I've seen this problem using both Xcode 10.1 and Xcode 10.2.
It appears to be a problem with Xamarin.UITest incorrectly parsing the JSON output of xcrun simctl list devices --json.

I think Xamarin.UITest.iOS.Instruments.GetSimctlSimulators is expecting the "devices" keys to contains spaces, which it splits them on. This may have been a previous behavior. However, simctl seems to be returning keys that use dashes instead. Thus, the simulator version cannot be determined.

For instance,

> xcrun simctl list devices --json

{
  "devices" : {
    "com.apple.CoreSimulator.SimRuntime.watchOS-5-1" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.tvOS-11-4" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.watchOS-5-2" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.iOS-11-4" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.iOS-12-0" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.watchOS-4-3" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.tvOS-12-0" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.iOS-12-2" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.watchOS-5-0" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.tvOS-12-1" : [
      ...
    ],
    "com.apple.CoreSimulator.SimRuntime.tvOS-12-2" : [
      ...
    ]
  }
}

[GeorgeDarakchiev]

Hi @agaluzzi ,

Thanks to pointing at this direction. I can confirm that the json is in different format:

After Xcode10.2 installed:
"devices" : {
"com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
{
"availability" : "(available)",
"state" : "Shutdown",
"isAvailable" : true,
"name" : "iPhone 5s",
"udid" : "9AC9332D-7DB0-4F2E-8615-61EE86E363CA",
"availabilityError" : ""
},

On machine with Xcode 10.1 installed:
"devices" : {
"iOS 12.1" : [
{
"state" : "Shutdown",
"isAvailable" : "YES",
"name" : "iPhone 5s",
"udid" : "2647F251-04A3-4DB8-8684-D66189869B42",
"availabilityError" : ""
},

So this seems to be intended change with Xcode 10.2 (inside the simctl):
https://openradar.appspot.com/radar?id=5058478808760320

I was able to get it back working like this:

1. Downgrading to Xcode 10.1 - at this point it was still the wrong json format.
2. Forcing simctl to reinstall by deleting the /Library/Developer directory. Important : This also deleted all of my simulators runtimes, so you ca try just deleting the subfolder that is related to the simctl (not tested).

@Oddj0b
When can we expect support for Xcode 10.2? Shouldn't you guys add the support when the Xcode becomes official or at least test with betas to make sure issues like this does not occur?

[GeorgeDarakchiev]

@Oddj0b and @patniko

Just to clarify why this is important:

Visual Studio 2019 (for Windows) requires Xcode 10.2 and therefore I should chose between VS 2019 or be able to run tests. Not been able to run and create tests is not an option for me and sadly I had to downgrade to Visual Studio 2017.
It's a shame that I am not able to use all of the goodies and Xamarin development improvements done in VS 2019 due to this issue.

[King-of-Spades]

The newest Xamarin.UITest dev releases have added Xcode 10.2 support, from this version or newer: https://www.nuget.org/packages/Xamarin.UITest/2.2.7.2002-dev