iOS test simulators fail to start on 20250811 macOS-15 build

[freakboy3742]

Description

As of the 20250811 macos-15-arm64 release, Xcode appears to be unable to launch iOS simulators as part of an XCUnit test suite.

I have a XCUnit test suite that has worked for some time. It runs happily using Xcode 16.4 and an iOS 18.6 simulator on my own machine. It has also run successfully on every other iOS micro versions. It also ran on Xcode 16.0 and iOS 18.0 on the previous macos-15-arm64 image.

I am aware of #12541 and the changes to the included iOS SDK runtimes. I have applied the recommended change to explicitly select Xcode 16.4 in my actions workflow.

However, as of the macOS image change change, when I run my test suite, I get errors indicating that Xcode is unable to start an iOS simulator.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

The problem has been observed in:

• https://github.com/pypa/cibuildwheel (https://github.com/pypa/cibuildwheel/actions/runs/16973833037/job/48125995960)
• Update cibuildwheel configuration to use current release. freakboy3742/pyspamsum#75 (https://github.com/freakboy3742/pyspamsum/actions/runs/17034048860/job/48282410568?pr=75)

Is it regression?

Yes; any build against the previous macOS-15-arm64 image.

Expected behavior

It should be possible to run an Xcode project that starts an iOS simulator using:

xcodebuild test -project localtest/iOSTestbed.xcodeproj -scheme iOSTestbed -destination 'platform=iOS Simulator,name=iPhone 16e'

Actual behavior

The test project builds, but then fails to start the simulator, with an error:

  Writing result bundle at path:
  	/private/var/folders/x7/ch5v91h56_zbvbd1y2f600dm0000gn/T/cibw-run-lfch6xkz/cp313-ios_arm64_iphonesimulator/testbed/20250818-071548.xcresult
  2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions Finished with error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices
  Domain: com.apple.CoreSimulator.SimError
  Code: 404
  --
  2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions com.apple.dt.IDERunOperationWorkerFinished {
      "operation_duration_ms" = 8;
      "operation_errorCode" = 404;
      "operation_errorDomain" = "com.apple.CoreSimulator.SimError";
      "operation_name" = "_IDEInstalliPhoneSimulatorWorker";
  }
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 elapsed -- Testing started completed.
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
  2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 sec, +9.957 sec -- end
  Testing failed:
  	iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices))

If I ssh into the machine and run the suite manually, I get the following error:

Writing result bundle at path:
        /Users/runner/work/pyspamsum/pyspamsum/localtest/20250818-055037.xcresult

2025-08-18 05:50:49.068 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: Finished with error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices
Domain: com.apple.CoreSimulator.SimError
Code: 404
User Info: {
    DVTErrorCreationDateKey = "2025-08-18 05:50:49 +0000";
    IDERunOperationFailingWorker = "_IDEInstalliPhoneSimulatorWorker";
}
--

2025-08-18 05:50:49.069 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: com.apple.dt.IDERunOperationWorkerFinished {
    "device_identifier" = "FC1D56A6-910E-4DDA-8DC7-776480505F9C";
    "device_model" = "iPhone17,5";
    "device_osBuild" = "26.0 (23A5308g)";
    "device_platform" = "com.apple.platform.iphonesimulator";
    "device_thinningType" = "iPhone17,5";
    "dvt_coredevice_version" = "477.29";
    "dvt_coresimulator_version" = 1047;
    "dvt_mobiledevice_version" = "1818.0.1";
    "launchSession_schemeCommand" = Test;
    "launchSession_state" = 1;
    "launchSession_targetArch" = arm64;
    "operation_duration_ms" = 11;
    "operation_errorCode" = 404;
    "operation_errorDomain" = "com.apple.CoreSimulator.SimError";
    "operation_errorWorker" = "_IDEInstalliPhoneSimulatorWorker";
    "operation_name" = IDERunOperationWorkerGroup;
    "param_debugger_attachToExtensions" = 0;
    "param_debugger_attachToXPC" = 0;
    "param_debugger_type" = 1;
    "param_destination_isProxy" = 0;
    "param_destination_platform" = "com.apple.platform.iphonesimulator";
    "param_diag_113575882_enable" = 0;
    "param_diag_MainThreadChecker_stopOnIssue" = 0;
    "param_diag_MallocStackLogging_enableDuringAttach" = 0;
    "param_diag_MallocStackLogging_enableForXPC" = 0;
    "param_diag_allowLocationSimulation" = 1;
    "param_diag_checker_tpc_enable" = 0;
    "param_diag_gpu_frameCapture_enable" = 3;
    "param_diag_gpu_shaderValidation_enable" = 0;
    "param_diag_gpu_validation_enable" = 1;
    "param_diag_guardMalloc_enable" = 0;
    "param_diag_memoryGraphOnResourceException" = 0;
    "param_diag_mtc_enable" = 0;
    "param_diag_queueDebugging_enable" = 1;
    "param_diag_runtimeProfile_generate" = 1;
    "param_diag_sanitizer_asan_enable" = 0;
    "param_diag_sanitizer_tsan_enable" = 0;
    "param_diag_sanitizer_tsan_stopOnIssue" = 0;
    "param_diag_sanitizer_ubsan_enable" = 0;
    "param_diag_sanitizer_ubsan_stopOnIssue" = 0;
    "param_diag_showNonLocalizedStrings" = 0;
    "param_diag_viewDebugging_enabled" = 0;
    "param_diag_viewDebugging_insertDylibOnLaunch" = 0;
    "param_install_style" = 2;
    "param_launcher_UID" = 2;
    "param_launcher_allowDeviceSensorReplayData" = 0;
    "param_launcher_kind" = 0;
    "param_launcher_style" = 0;
    "param_launcher_substyle" = 0;
    "param_runnable_appExtensionHostRunMode" = 0;
    "param_runnable_productType" = "com.apple.product-type.application";
    "param_structuredConsoleMode" = 0;
    "param_testing_launchedForTesting" = 1;
    "param_testing_suppressSimulatorApp" = 1;
    "param_testing_usingCLI" = 0;
    "sdk_canonicalName" = "iphonesimulator18.5";
    "sdk_osVersion" = "18.5";
    "sdk_variant" = iphonesimulator;
}
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 elapsed -- Testing started completed.
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 sec, +12.606 sec -- end
Testing failed:
        iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices))

The only thing that stands out in this error log is the reference to Xcode 26; given than I'm explicitly using Xcode 16.4, it's not clear to me why iOS 26 would be involved.

Repro steps

The test suites that are affected are all using CPython's iOS testbed, by way of cibuildwheel.

The problem can be reproduced by downloading this tarball, unpacking it, then running:

$ curl -L https://github.com/beeware/Python-Apple-support/releases/download/3.13-b10/Python-3.13-iOS-support.b10.tar.gz -o Python-3.13-iOS-support.b10.tar.gz
$ tar zxvf Python-3.13-iOS-support.b10.tar.gz
$ python3.13 testbed clone localtest
$ python3.13 localtest run -- test test_platform

This will run a minimal subset of the Python test suite in an iPhone 16e simulator.

This sequence of commands works locally, but fails in the macos-15-arm64 environment as of the 20250811 update, regardless of what Xcode version has been set as the default.

localtest/__main__.py contains the code to launch Xcode and stream logs from the simulator that is created. The code starting xcodebuild is around L248 of that script.

[badredaha]

Hi,

same error here +1

[archita105]

Hi @freakboy3742 - Thank you for bringing this issue to our attention. We will look into this issue and will update you .

[gruselhaus]

@archita105 this is directly connected to #12768.

This completely broke our CI.

[erik-bershel]

Hey @gruselhaus!

The mentioned PR cannot affect the problem mentioned by the topic starter, since it belongs to a different version of the image (Intel-based macOS-15) and the changes have not yet been delivered.

[erik-bershel]

Hey @freakboy3742!

The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling xcodebuild, judging by the general log. If the test suite recognizes Xcode 26 components as belonging to Xcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.