Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to find iOS Simulators (Post Xcode 10.2 install) #2691

Closed
msaps opened this Issue Jan 25, 2019 · 45 comments

Comments

Projects
None yet
@msaps
Copy link

msaps commented Jan 25, 2019

  • carthage install method: [ ] .pkg, [x] homebrew, [ ] source
  • which carthage: /usr/local/bin/carthage
  • carthage version: 0.31.2
  • xcodebuild -version: 10.1

After installing Xcode 10.2, Carthage fails to build any project due to Could not find any available simulators for iOS. I've tried a number of different things to resolve it:

  • Build a project in Xcode 10.2 on a simulator (suggested in #2690)
  • Use Xcode.app and Xcode-beta.appfor xcode-select.
  • Reset all my simulators etc.

I think the problem is the keys that are in the output of simctl. Looking through Simulator.swift in CarthageKit I can see that it expects to find a platform prefix for the key (i.e. iOS) - my keys however are formatted like this: com.apple.CoreSimulator.SimRuntime.iOS-12-1.

Is this a problem with my installation? Or is it something that has changed in Xcode 10.2 and should be supported in Carthage?

Example xcrun simctl list devices --json output:

"com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
      {
        "availability" : "(available)",
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 5s",
        "udid" : "9AC9332D-7DB0-4F2E-8615-61EE86E363CA",
        "availabilityError" : ""
      },
      {
        "availability" : "(available)",
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 6",
        "udid" : "B1DA5CFF-D918-467B-8E49-04B6172B9456",
        "availabilityError" : ""
      }
]
@blender

This comment was marked as outdated.

Copy link
Member

blender commented Jan 25, 2019

Are you installing Xcode via MAS or developer portal?

I don't think it's and Xcode 10.2 specific issue but something related to your installation. I have no idea however why something like this would happen.

@blender blender added the question label Jan 25, 2019

@msaps

This comment was marked as outdated.

Copy link
Author

msaps commented Jan 25, 2019

@blender I run the stable version of Xcode through the MAS - I'll try nuking everything and see if that resolves it. Also maybe avoid the MAS in future 🤣

@blender

This comment was marked as outdated.

Copy link
Member

blender commented Jan 25, 2019

Yeah, I personally don't use MAS and I have never so far encountered issues like this. I don't want to correlate the two but...

@mattgallagher

This comment has been minimized.

Copy link

mattgallagher commented Jan 25, 2019

This type of problem has affected Carthage before. It is not your fault: Carthage needs to be updated to handle the new JSON format used by the simctl that ships with the Xcode 10.2 beta.

FYI, after encountering the same problem, the issue persisted for me, even after reverting to Xcode 10.1 and deleting all Carthage build artifacts and caches.

I needed to run:

xcrun simctl delete unavailable

to delete unreadable simulators and reset the JSON format.

EDIT I think my analysis here was flawed. I had also deleted the /Library/Developer directory at the same time, forcing a re-install of the Xcode 10.1 simctl tool. I think this was the true fix for me.

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

Tried nuking my entire Xcode installation (including ~/Library/Developer/,~/Library/Caches/com.apple.dt.Xcode and the CLI tools), reinstalling 10.1 (from the dev portal) and the Simulator keys are still the same.

@mattgallagher also tried xcrun simctl delete unavailable to no avail, both before and after reinstalling Xcode.

I'm not sure what causes this format to occur but does seem like something that might be useful for Carthage to handle? What do you think @blender?

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

I don't think it's a carthage problem honestly.

So @msaps you have installed 10.1, ran xcrun simctl delete unavailable and now the problem is solved?

This seems to be like it's triggered by installing 10.2.

I think the right way here is to open a radar and probably add yet another ad-hoc patch for a non final version of Xcode...

@taher-mosbah

This comment has been minimized.

Copy link

taher-mosbah commented Jan 25, 2019

Same problem for me after installing 10.2 via xcversion and switching to it using xcversion select "10.2" , switching back to 10.1 did not help.

xcrun simctl delete unavailable did not solve the issue.

What can we do to help ?

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

try checking what version of simclt is actually being invoked

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

@blender unfortunately not. Completely removed my installation, reinstalled 10.1 and the problem still exists. xcrun simctl delete unavailable didn't seem to do much and recreating the simulators with Fastlane fastlane snapshot reset_simulators also didn't work (both before and after reinstalling Xcode).

simctl version: @(#)PROGRAM:simctl PROJECT:CoreSimulator-587.24

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

One potentially interesting point is that I still have the 10.2 Simulator display names in my simulators list (Believe the R and S are rendered differently in <10.2).

screenshot 2019-01-25 at 12 06 08

Unless I missed something stupid, I did remove every Xcode cache/directory I could think of. Had to reinstall the Command Line Tools etc.

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

xcrun simctl --version on Xcode 10.1 returns @(#)PROGRAM:simctl PROJECT:CoreSimulator-581.2

Xcode 10.2 Beta yields:

@(#)PROGRAM:simctl PROJECT:CoreSimulator-587.24

so @msaps you're still running the new simctl

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

I see two device outputs formats currently from xcrun simctl list devices --json

one has the devices under the key in the format like platform <version> and the other one has the format com.apple.CoreSimulator.SimRuntime.<platform>-<major_version>-<minor_version>

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

I only get the verbose format.

How can my simctl version be out of sync with the current Xcode version? 🤔

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

@msaps do you have 10.1 CLI tools installed?

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

Open 10.1 and run a project on the simulator.

Now xcrun simctl list devices --json will output the format the carthage understands

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

@blender

CLI Tools look okay.
screenshot 2019-01-25 at 12 32 38

Running on the 10.1 Sim makes no difference to my simctl list output. I am really confused as to how I have the 10.2 version of simctl with no trace of it installed on my machine.

@apoltix

This comment has been minimized.

Copy link

apoltix commented Jan 25, 2019

I am having the same issue. I've checked out Carthage 0.31.2 and run it in Xcode 10.1 to see where it goes wrong, and have found the Simulator class checks the platform's name by prefix (e.g. "iOS"), but the prefix has (seemingly) changed with Xcode 10.2 so it's now "com.apple.CoreSimulator.SimRuntime.iOS-12-2".

After changing hasPrefix to contains in Simulator.swift's selectAvailableSimulator function, it correctly reads the simulators. For some reason they're all marked as isAvailable = false, so I'm looking into that now.

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

I can reliably switch versions of simctl and reset the output by doing:

  • sudo xcode-select -s /Applications/Xcode_10.1/Contents/Developer
  • Running any app that boots the simulator
  • xcrun simctl list devices --json
@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 25, 2019

@msaps did you xcode-select 10.1 ?

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

@blender yeah I was on 10.1 - switching between that and 10.2 didn't seem to make any difference to what simctl my machine was using weirdly.

@apoltix

This comment has been minimized.

Copy link

apoltix commented Jan 25, 2019

So I've managed to run Carthage 0.31.2 successfully with a single code patch, the aforementioned change of hasPrefix to contains, building and running from Xcode 10.1 but setting the environment variable DEVELOPER_DIR to the Xcode 10.2 beta developer path, e.g. /Applications/Xcode-beta.app/Contents/Developer. This makes Carthage use the correct xcrun so the simulators are available.

@msaps

This comment has been minimized.

Copy link
Author

msaps commented Jan 25, 2019

@apoltix yeah I'm currently using a locally patched version of Carthage as well.

@russbishop

This comment has been minimized.

Copy link

russbishop commented Jan 25, 2019

When Xcode is updated CoreSimulator is updated. CoreSimulator always rolls forward so it will be the version installed by the newest Xcode you ever launched. simctl is now bundled with CoreSimulator, that's why you are seeing the newer version.

@ikesyo ikesyo closed this Jan 26, 2019

@ikesyo ikesyo reopened this Jan 26, 2019

@drekka

This comment has been minimized.

Copy link

drekka commented Jan 26, 2019

I definitely have the issues where Carthage cannot see any simulators. But also noticed that the Components tab in Xcode 10.2 beta preferences is empty and won't download any.

It sounds like a lot of people are trying to back up to 10.1. I'd like to get 10.2 going instead.

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 26, 2019

@drekka You can patch carthage to do just that like so #2691 (comment)

If you're interested in a little (pointless?) investigation read forward

I have done a little investigation following what @russbishop has said and I found the following.

I am currently using Xcode 9.2 and Xcode 10.2

in Xcode 9.2 simctl is at /Applications/Xcode_9.2.app/Contents/Developer/usr/bin/simctl

this version links to

/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/CoreSimulator (compatibility version 1.0.0, current version 494.33.0)
@rpath/SimulatorKit.framework/Versions/A/SimulatorKit (compatibility version 1.0.0, current version 494.33.0)

in Xcode 10.2 instead simctl is at /Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl

this version links to

/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/CoreSimulator (compatibility version 1.0.0, current version 587.24.0)
/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/Frameworks/CoreSimulatorUtilities.framework/Versions/A/CoreSimulatorUtilities (compatibility version 1.0.0, current version 587.24.0)

Now, when I xcode-select Xcode 9.2 the "old" (let's call it 9.2 version) version of simctl is called from xcrun and the opposite when I xcode-select Xcode 10.2

Since the linked version of CoreSimulator is the same and simctl 9.2 and simctl 10.2 both use a common subset of the CoreSimulator API (the other frameworks look inconsequential) I believe this is a bug (or an arbitrary change) in simctl 10.2. In fact unsurprisingly if I replace simctl 10.2 with a simlink to simctl 9.2 an xcode-select Xcode 10.2 carthage works just fine.

simctl 10.2 CoreSimulator API:

$ objdump -bind /Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl | grep CoreSimulator
/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl:	file format Mach-O 64-bit x86-64
__DATA   __got              0x1000432F8 pointer         0 CoreSimulatorUtilities _SimPrefKeyRuntimeImageCacheMaxSizeGB
__DATA   __got              0x100043300 pointer         0 CoreSimulatorUtilities _SimPrefKeyRuntimeImageCachePath
__DATA   __objc_const       0x10004A820 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_const       0x10004AC28 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x10004DCD8 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x10004DD68 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimNSPasteboard
__DATA   __objc_classrefs   0x10004DD88 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimPasteboardItem
__DATA   __objc_classrefs   0x10004DD58 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimServiceContext
__DATA   __got              0x100043268 pointer         0 CoreSimulator    _SimAudioHostUseSystemDefaultDeviceUID
__DATA   __got              0x100043270 pointer         0 CoreSimulator    _SimDeviceBootKeyDisabledJobs
__DATA   __got              0x100043278 pointer         0 CoreSimulator    _SimDeviceBootKeyEnvironment
__DATA   __got              0x100043280 pointer         0 CoreSimulator    _SimDeviceBootKeyPersist
__DATA   __got              0x100043288 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyArguments
__DATA   __got              0x100043290 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyEnvironment
__DATA   __got              0x100043298 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyStandardErrPath
__DATA   __got              0x1000432A0 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyStandardOutPath
__DATA   __got              0x1000432A8 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyWaitForDebugger
__DATA   __got              0x1000432B0 pointer         0 CoreSimulator    _SimDeviceNotificationType_BootStatus
__DATA   __got              0x1000432B8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyArguments
__DATA   __got              0x1000432C0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyBinPref
__DATA   __got              0x1000432C8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyEnvironment
__DATA   __got              0x1000432D0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStandalone
__DATA   __got              0x1000432D8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStderr
__DATA   __got              0x1000432E0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStdin
__DATA   __got              0x1000432E8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStdout
__DATA   __got              0x1000432F0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyWaitForDebugger
__DATA   __got              0x1000433A0 pointer         0 CoreSimulator    _kSimDeviceNotification_NewBootStatus

simctl 9.2 CoreSimulator API

$ objdump -bind /Applications/Xcode_9.2.app/Contents/Developer/usr/bin/simctl | grep CoreSimulator
__DATA   __objc_const       0x10002F850 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x100030D00 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x100030D58 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimNSPasteboard
__DATA   __objc_classrefs   0x100030D80 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimPasteboardItem
__DATA   __objc_classrefs   0x100030D40 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimServiceContext
__DATA   __got              0x10002A028 pointer         0 CoreSimulator    _SimAudioHostUseSystemDefaultDeviceUID
__DATA   __got              0x10002A030 pointer         0 CoreSimulator    _SimDeviceNotificationType_BootStatus
__DATA   __got              0x10002A0C0 pointer         0 CoreSimulator    _kSimDeviceNotification_NewBootStatus
@mdiep

This comment has been minimized.

Copy link
Member

mdiep commented Jan 26, 2019

I think we need to suck it up and update our use of the JSON format.

These seemingly random JSON changes from the simulator team are extremely disappointing. This is 2nd Xcode release in a row that includes a breaking change to the JSON API—with no mention in the release notes and seemingly with no reason for it.

I assumed that the point of JSON output was easier consumption by tooling. But these breaking changes make it worse for tooling, not better. It makes me regret switching to the JSON output at all. We would have been better off parsing the normal output since it hasn't changed.

@Nonnus

This comment has been minimized.

Copy link

Nonnus commented Jan 26, 2019

@mdiep from your words I assume we need to wait for a Carthage update to fix the issue and allow use of Xcode 10.2?

@mdiep

This comment has been minimized.

Copy link
Member

mdiep commented Jan 26, 2019

Yup!

@hamchapman hamchapman referenced this issue Jan 27, 2019

Closed

Xcode 10.2 Error #8458

1 of 1 task complete
@drekka

This comment has been minimized.

Copy link

drekka commented Jan 27, 2019

Thanks @blender I've downloaded the Carthage source with a view to taking a look, but I'm having problems working with it. Could be an issue with 10.2 as well. Who knows.

@qnoid

This comment has been minimized.

Copy link

qnoid commented Jan 27, 2019

@mdiep curious if you you know this to be as intended and not a regression. If so, is that a change in the spec of simctl?

Emphasis mine. According to simctl, the list command is used to "List available devices, device types, runtimes, or device pairs.".

AFAICS the devices are missing. Are devices meant to be inferred by the runtimes starting with Xcode 10.2?

Could you please elaborate @russbishop?

@dive

This comment has been minimized.

Copy link

dive commented Jan 27, 2019

Btw, it seems that Apple removed all previous simulator runtimes support from Xcode 10.2b. Perhaps, due to missing dynamically linked libraries for the Swift standard library. I tried to link them manually, but it didn't work to me. I didn't try hard though.
I mean, you'll see the following information if you switch to Xcode 10.2b with xcode-select and try to list the simulators:

$ xcrun simctl list devices --json | grep -A16 12.1
    "com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
      {
        "availability" : "(unavailable, runtime profile not found)",
        "state" : "Shutdown",
        "isAvailable" : false,
        "name" : "iPhone 5s",
        "udid" : "DDD36346-A76F-42E8-80F4-6F11E1EE4BEB",
        "availabilityError" : "runtime profile not found"
      },
      {
        "availability" : "(unavailable, runtime profile not found)",
        "state" : "Shutdown",
        "isAvailable" : false,
        "name" : "iPhone 6",
        "udid" : "21794717-BC89-45E4-9F57-8CF9D14A87D1",
        "availabilityError" : "runtime profile not found"
      },
...

No runtime for com.apple.CoreSimulator.SimRuntime.iOS-12-1 at all.

@blender

This comment has been minimized.

Copy link
Member

blender commented Jan 28, 2019

🥁

Apple Developer Relations
Please know that our engineering team has determined that this issue behaves as intended based on the information provided.
In order to better support pinning, this was the least disruptive option.

@qnoid

This comment has been minimized.

Copy link

qnoid commented Jan 28, 2019

What do they mean by pinning?

@dive

This comment has been minimized.

Copy link

dive commented Jan 29, 2019

I can reliably switch versions of simctl and reset the output by doing:

  • sudo xcode-select -s /Applications/Xcode_10.1/Contents/Developer
  • Running any app that boots the simulator
  • xcrun simctl list devices --json

@blender, hmm, this solution doesn't work to me.

Since the linked version of CoreSimulator is the same and simctl 9.2 and simctl 10.2 both use a common subset of the CoreSimulator API (the other frameworks look inconsequential) I believe this is a bug (or an arbitrary change) in simctl 10.2. In fact unsurprisingly if I replace simctl 10.2 with a simlink to simctl 9.2 an xcode-select Xcode 10.2 carthage works just fine.

So, the only way is to re-link simctl explicitly? Even in case when you have Xcode 10.1 and Xcode 10.2?

@shinma

This comment has been minimized.

Copy link

shinma commented Jan 29, 2019

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.

OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

@taher-mosbah

This comment has been minimized.

Copy link

taher-mosbah commented Jan 29, 2019

Now that #2693 is merged does it make sense to have a release ?
or should we wait for finale version of Xcode 10.2 ?

@Nonnus

This comment was marked as resolved.

Copy link

Nonnus commented Jan 29, 2019

hopefully we won't have to wait for any final version of Xcode 10.2 as this would be terribly disruptive to ongoing testing and development that should be done before the final release

@yapiskan

This comment has been minimized.

Copy link

yapiskan commented Jan 29, 2019

This issue is currently breaking our buddybuild CI builds. Would be great if there is a release.

@jdhealy

This comment has been minimized.

Copy link
Member

jdhealy commented Jan 29, 2019

Yes, we plan to have a release well before the final version of Xcode 10.2…

@mvnevesbrb

This comment has been minimized.

Copy link

mvnevesbrb commented Jan 30, 2019

Please, do you have a solution for this problem ? Or can you point how can I compile the source, because some packages :

error: dependency graph is unresolvable; found these conflicting requirements:

Dependencies:
https://github.com/jdhealy/PrettyColors.git @ 5.0.0..<6.0.0
https://github.com/thoughtbot/Curry.git @ 4.0.0..<5.0.0
'Carthage' ... error: product dependency 'Curry' not found
'Carthage' ... error: product dependency 'PrettyColors' not found
'Carthage' ... error: product dependency 'Curry' not found

@mvnevesbrb

This comment has been minimized.

Copy link

mvnevesbrb commented Jan 30, 2019

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.

OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

how did you achieve to compile the code, can you help?

@shinma

This comment has been minimized.

Copy link

shinma commented Jan 30, 2019

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.
OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

how did you achieve to compile the code, can you help?

  • git checkout 0.31.2
  • make boostrap
  • open Carthage.workspace
  • build using Xcode 10.1
  • make install to replace brew version (after fix is released make uninstall and brew upgrade)
@giginet

This comment was marked as outdated.

Copy link
Member

giginet commented Feb 1, 2019

After merging #2693 and #2696 , I tried on my project.
It works both on Xcode 10.1 and 10.2.

Could anyone release a newer version? @Carthage/carthage

@jdhealy jdhealy removed the question label Feb 1, 2019

@jdhealy

This comment has been minimized.

Copy link
Member

jdhealy commented Feb 1, 2019

Fixes released with https://github.com/Carthage/Carthage/releases/tag/0.32.0

Edit: Homebrew binary bottle released: 0.32.0.

Thanks @kmcbride and @DavidBrunow for submitting fixes!

Thank you to @giginet, @taher-mosbah, @ikesyo, and @blender for reviewing pull requests!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.