xctool cannot run tests against the "iPhone Retina (4-inch 64-bit)" simulator

[fpotter]

When passing -destination "name=iPhone Retina (4-inch 64-bit)" to xcodebuild ... test, xcodebuild will run tests as x86_64. xctool needs to do the same.

To fix, we need do a few things...

• Make otest-query + otest-shim Universal binaries with i386 + x86_64 flavors.

• Find a way to determine if a destination value refers to 64-bit or 32-bit arch. It looks like we can examine the plist for the named device (e.g. /Applications/Xcode.app/Contents/Developer/Library/PrivateFrameworks/SimulatorHost.framework/Versions/A/Resources/Devices/iPhone (Retina 4-inch 64-bit).deviceinfo/Info.plist), and look for wordSize=64 in the dictionary.

• When running logic tests, we need to force the x86_64 arch via -[NSConcreteTask setPreferredArchitectures:].

• When running applications tests, we need to launch the simulator in the correct 64-bit mode.

  When watching Xcode.app post distributed notifications to the simulator, we can see that it's passing arch=x86_64 and deviceInfo=iPhone Retina (4-inch 64-bit) in the com.apple.iphonesimulator.startSession message (see https://gist.github.com/fpotter/6941760). xctool needs to do the same.

  This will be a little tricky. xctool uses iPhoneSimulatorRemoteClient.framework to interact with the sim, but it doesn't expose a way to set the arch or deviceInfo fields of that distribution notification. We'll have to trick it into doing so, probably with some swizzling.

[ryanrhee]

Using DVTiPhoneSimulatorRemoteClient.framework expose setting the arch, right?

[ryanrhee]

A PR for building universal binaries is up: #211
This PR also uses -[NSConcreteTask setPreferredArchitectures:].

[ryanrhee]

A PR for using * DVTiPhoneSimulatorRemoteClient.framework* is up: #212

[ryanrhee]

@fpotter I think we can set the deviceInfo without using the distributed notification using some CF functions: CFPreferencesSetAppValue() and CFPreferencesAppSynchronize(). We just need to set the value for the SimulatedDevice key to be something like iPhone Retina (4-inch 64-bit). We can then look up the wordsize for the particular device to determine architecture, and then use the -[NSConcreteTask setPreferredArchitectures:] to fork a process that runs the simulator in the correct architecture.

If this works, this means we don't have to fight with DVTiPhoneSimulatorRemoteClient.framework and also don't have to manually broadcast NSDistributedNotification's. Yay!

[ryanrhee]

Check out the referenced commit -- It works! (Although it's pretty dirty ...)
Starting at the directory xctool/xctool-tests/TestData/TestWorkspace-Library, I'm able to do the following commands and see different simulators each time (with passing tests, too!):

../../../../xctool.sh --workspace TestWorkspace-Library.xcworkspace --scheme TestProject-Library -sdk iphonesimulator -destination "name=iPhone Retina (3.5-inch)" test -only TestProject-LibraryTests2
sleep(3)
killall iPhone\ Simulator
../../../../xctool.sh --workspace TestWorkspace-Library.xcworkspace --scheme TestProject-Library -sdk iphonesimulator -destination "name=iPhone Retina (4-inch 64-bit)" test -only TestProject-LibraryTests2

I don't know if this is a solution we want to pursue, but it's an option.

[ryanrhee]

@fpotter for logic tests, when do we force 64-bit? (Is the -destination arg still relevant for logic tests? I kind of assumed that they're only used for application tests.)

[ryanrhee]

#217 enables 64-bit application tests.

[fpotter]

Yeah, I think destination will have to be used for logic tests, too - we'll look at it to decide whether or not to force 64-bit.

You could double check with ... xcodebuild ... -destination "name=iPhone Retina (4-inch, 64-bit)" test for some project with logic tests, and make sure it's running the 64-bit flavor.

[ryanrhee]

Oh man. You scared me for a sec. Fortunately, the name is actually iPhone Retina (4-inch 64-bit). No comma. Hooray! (Otherwise, we would have had to do much more complex logic when parsing the destination.)

More importantly, it looks like xcodebuild also blindly splits on the comma. :)

[ryanrhee]

@fpotter it seems like running xcodebuild ... -destination "name=iPhone Retina (4-inch 64-bit)" test on a library actually launches a simulator. (Calling xctool with the same args don't launch a simulator.) I actually also tried xcodebuild ... -sdk iphonesimulator test, and got the same result. Why does xcodebuild launch a simulator on logic tests? (And how does xctool not launch a simulator?)

[ryanrhee]

Yeah, the -destination does set the arch. We'll have to force the arch depending on the -destination.

[fpotter]

closing - i think @ryanrhee land this already.