[ci] Xharness fails to install application to run Device Tests and UITests

[rmarinho]

Description

We are getting a lot of failures installing the application on iOS 16.4 simulators.

https://dev.azure.com/xamarin/public/_build/results?buildId=106273&view=logs&j=8d066757-897f-520b-6555-d94730b90fe5&t=1844224e-ccad-5261-c882-ccacb1d19962&l=353

Affected platforms

iOS

Relevant log output

fail: Failed to install the application
info: Cleaning up the failed installation from 'iPhone Xs (iOS 16.4) - created by XHarness'
info: Uninstalling the application 'com.microsoft.maui.graphics.devicetests' from 'iPhone Xs (iOS 16.4) - created by XHarness'
dbug: 
dbug: Running /Applications/Xcode_15.2.0.app/Contents/Developer/usr/bin/simctl uninstall 69691407-44A7-40E2-A275-2469456DC1C4 com.microsoft.maui.graphics.devicetests
dbug: An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=405):
dbug: Unable to lookup in current state: Shutdown
dbug: Process simctl exited with 149
fail: Failed to uninstall the app bundle! Check logs for more details!
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)

https://dev.azure.com/xamarin/public/_build/results?buildId=106273&view=logs&j=f419283a-3739-5707-3f25-e8ceb2a73005&t=58eb9ca2-cfb4-59be-a202-bd3054af645a&l=354

list-ios-simulator-64_16.4-20240123_085457.log
test-ios-simulator-64_16.4.log

[rmarinho]

This seems to happen also on UITests https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=8963003&view=logs&j=bc1d4079-a306-5df1-792a-3c37420504d1&t=e7f021c7-089d-579f-93b3-6a2f5e8542b2&l=1365

dbug: Preemptive uninstallation of application failed
info: Installing application 'MauiCompatibilityControlGallery' on 'iPhone Xs (iOS 16.4) - created by XHarness'
dbug: Installing '/Users/builder/azdo/_work/3/s/src/Compatibility/ControlGallery/src/iOS/bin/Debug/net8.0-ios/iossimulator-x64/Microsoft.Maui.Controls.Compatibility.ControlGallery.iOS.app' to 'iPhone Xs (iOS 16.4) - created by XHarness' (90.53 MB)
dbug: 
dbug: Running /Users/builder/.nuget/packages/microsoft.dotnet.xharness.cli/9.0.0-prerelease.24066.3/tools/net8.0/any/../../../runtimes/any/native/mlaunch/bin/mlaunch --sdkroot /Applications/Xcode_15.2.0.app --device=:v2:udid=A71B1542-677B-448E-8E00-17EBDAA2B242 --installsim /Users/builder/azdo/_work/3/s/src/Compatibility/ControlGallery/src/iOS/bin/Debug/net8.0-ios/iossimulator-x64/Microsoft.Maui.Controls.Compatibility.ControlGallery.iOS.app -v -v
dbug: Using Xcode 15.2 found in /Applications/Xcode_15.2.0.app
dbug: Xamarin.Hosting: Xamarin.Hosting
dbug: Xamarin.Hosting:     Version: f8f70b23a4 (refs/heads/main)
dbug: Xamarin.Hosting:     Xcode: /Applications/Xcode_15.2.0.app
dbug: Xamarin.Hosting:     Xcode Version: 15.2
dbug: Xamarin.Hosting:     Verbosity: 2
dbug: Xamarin.Hosting: Booting iPhone Xs (iOS 16.4) - created by XHarness...
dbug: Xamarin.Hosting: Failed to boot iPhone Xs (iOS 16.4) - created by XHarness: Unable to boot the Simulator..
dbug: Xamarin.Hosting: Will try to continue anyway.
dbug: Xamarin.Hosting: Installing on iOS 16.4 (20E247) - iPhone Xs by executing 'xcrun simctl install A71B1542-677B-448E-8E00-17EBDAA2B242 /Users/builder/azdo/_work/3/s/src/Compatibility/ControlGallery/src/iOS/bin/Debug/net8.0-ios/iossimulator-x64/Microsoft.Maui.Controls.Compatibility.ControlGallery.iOS.app'
dbug: An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=405):
dbug: Unable to lookup in current state: Shutdown
dbug: error HE0046: Failed to install the app 'com.microsoft.mauicompatibilitygallery' on the device 'iOS 16.4 (20E247) - iPhone Xs': simctl returned exit code 149
dbug: Process mlaunch exited with 1
fail: Failed to install the application
info: Cleaning up the failed installation from 'iPhone Xs (iOS 16.4) - created by XHarness'
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)
Looking for simulator: XHarness iosversion 16.4

[ghost]

We've added this issue to our backlog, and we will work to address it as time and resources allow. If you have any additional information or questions about this issue, please leave a comment. For additional info about issue management, please read our Triage Process.

[PureWeen]

@rmarinho @mattleibow I chatted with @mandel-macaque about this and he was saying that we shouldn't keep recreating and destroying the devices with every run. Doing that so often causes it to go haywire sometimes.

Can we not create and destroy it every time? Any specific reason we are doing that?