New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xctool fails to launch simulator and install application when running tests against XCode 6 Beta 5 #404

Closed
obuseme opened this Issue Aug 12, 2014 · 50 comments

Comments

Projects
None yet
@obuseme

obuseme commented Aug 12, 2014

Running application tests for iOS using xctool with XCode 6 Beta 5 fails. This worked with XCode 6 Beta 4.

export DEVELOPER_DIR="/Applications/Xcode6-Beta4.app/Contents/Developer"
<path to xctool>/xctool.sh -workspace <workspace>.xcworkspace -scheme <schema> test -sdk iphonesimulator8.0

Result:
[Info] Collecting info for testables... (1542 ms)
  run-test appTests.xctest (iphonesimulator8.0, application-test)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (51 ms)
    [Info] Preparing test environment failed; will retry 2 more times (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (120 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/jenkins/Library/Developer/CoreSimulator/Devices/BD4FCA64-F2E1-4A49-9DBF-A3F1A5E6E897/data" (103 ms)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (15 ms)
    [Info] Preparing test environment failed; will retry 1 more time (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (120 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/jenkins/Library/Developer/CoreSimulator/Devices/BD4FCA64-F2E1-4A49-9DBF-A3F1A5E6E897/data" (120 ms)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (12 ms)
    [Warning] Preparing test environment failed. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
--------------------------------------------------------------------------------
There was a problem starting the test bundle: Simulator 'iPhone 4s' was not prepared: Failed for unknown reason.
----|---------------------------------------------------------------------------
    X -[TEST_BUNDLE FAILED_TO_START] (0 ms) (0)
      -[APICompletionHandlerTests testBlockCallsLogoutAppropriately]

...cut out all other tests fail...

@LegNeato

This comment has been minimized.

Show comment
Hide comment
@LegNeato

LegNeato Aug 12, 2014

Contributor

We'll take a look

Contributor

LegNeato commented Aug 12, 2014

We'll take a look

@Pretz

This comment has been minimized.

Show comment
Hide comment
@Pretz

Pretz Aug 14, 2014

Contributor

I'm experiencing this error specifically when running from an ssh session. It doesn't happen when I run from Terminal locally. Maybe Apple changed something about how xcodebuild gathers information about the current environment?

Contributor

Pretz commented Aug 14, 2014

I'm experiencing this error specifically when running from an ssh session. It doesn't happen when I run from Terminal locally. Maybe Apple changed something about how xcodebuild gathers information about the current environment?

@LegNeato

This comment has been minimized.

Show comment
Hide comment
@LegNeato

LegNeato Sep 3, 2014

Contributor

Can you try with the latest HEAD of master?

Contributor

LegNeato commented Sep 3, 2014

Can you try with the latest HEAD of master?

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 3, 2014

I have a similar problem with xcodebuild from Xcode 6 beta 7. It fails to run the tests when running from an ssh session.

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

This happens with Xcode 6 beta 7 also. I did report this to apple a month ago. But no fixes so far.

xfreebird commented Sep 3, 2014

I have a similar problem with xcodebuild from Xcode 6 beta 7. It fails to run the tests when running from an ssh session.

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

This happens with Xcode 6 beta 7 also. I did report this to apple a month ago. But no fixes so far.

@ExtremeMan

This comment has been minimized.

Show comment
Hide comment
@ExtremeMan

ExtremeMan Sep 3, 2014

Contributor

There are several hacks in xctool that are intended to workaround this issue. @xfreebird could you confirm that this reproduces in the latest xctool as well? If yes, steps to reproduce would be helpful.
Thanks.

Contributor

ExtremeMan commented Sep 3, 2014

There are several hacks in xctool that are intended to workaround this issue. @xfreebird could you confirm that this reproduces in the latest xctool as well? If yes, steps to reproduce would be helpful.
Thanks.

@LegNeato

This comment has been minimized.

Show comment
Hide comment
@LegNeato

LegNeato Sep 3, 2014

Contributor

To run the simulator you need to have an active GUI login session (ugh, yay Apple). SSH sessions don't get all the launchd stuff. See:

https://developer.apple.com/library/mac/technotes/tn2083/_index.html#//apple_ref/doc/uid/DTS10003794-CH1-SECTION9

and

travis-ci/travis-ci#1630

So I don't think this is a real bug (are you sure you weren't logged into the GUI when previously testing?), or if it is it is a bug in Xcode.

Contributor

LegNeato commented Sep 3, 2014

To run the simulator you need to have an active GUI login session (ugh, yay Apple). SSH sessions don't get all the launchd stuff. See:

https://developer.apple.com/library/mac/technotes/tn2083/_index.html#//apple_ref/doc/uid/DTS10003794-CH1-SECTION9

and

travis-ci/travis-ci#1630

So I don't think this is a real bug (are you sure you weren't logged into the GUI when previously testing?), or if it is it is a bug in Xcode.

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 3, 2014

In my case I already have an active GUI login session for the same user logged in using ssh. This does work with Xcode 5, but not with Xcode 6 beta 7.

I'm using xcodebuild directly.

xfreebird commented Sep 3, 2014

In my case I already have an active GUI login session for the same user logged in using ssh. This does work with Xcode 5, but not with Xcode 6 beta 7.

I'm using xcodebuild directly.

@LegNeato

This comment has been minimized.

Show comment
Hide comment
@LegNeato

LegNeato Sep 3, 2014

Contributor

Ah. We are still on Beta 6, were looking to update tomorrow to 7. Sounds like an Apple bug we probably can't work around :-/

Contributor

LegNeato commented Sep 3, 2014

Ah. We are still on Beta 6, were looking to update tomorrow to 7. Sounds like an Apple bug we probably can't work around :-/

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 3, 2014

@ExtremeMan I tested with the xctool from master HEAD, you can see the compele log here http://pastebin.com/WW8fQuxz

It fails to start the simulator.

How to reproduce:

  1. Start an active GUI login session on the mac (with Xcode 6 beta 5 or beta 6 or beta 7)
  2. From a different station connect through ssh
  3. Run tests from command line

What happens:
Tests fail to run due to simulator.

What is expected:
Tests run in the simulator successfully

xfreebird commented Sep 3, 2014

@ExtremeMan I tested with the xctool from master HEAD, you can see the compele log here http://pastebin.com/WW8fQuxz

It fails to start the simulator.

How to reproduce:

  1. Start an active GUI login session on the mac (with Xcode 6 beta 5 or beta 6 or beta 7)
  2. From a different station connect through ssh
  3. Run tests from command line

What happens:
Tests fail to run due to simulator.

What is expected:
Tests run in the simulator successfully

@ExtremeMan

This comment has been minimized.

Show comment
Hide comment
@ExtremeMan

ExtremeMan Sep 3, 2014

Contributor

@xfreebird does xcodebuild work in the same situation when xctool fails?

Contributor

ExtremeMan commented Sep 3, 2014

@xfreebird does xcodebuild work in the same situation when xctool fails?

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 3, 2014

No, it fails too.

Something like:

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

xfreebird commented Sep 3, 2014

No, it fails too.

Something like:

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

@ExtremeMan

This comment has been minimized.

Show comment
Hide comment
@ExtremeMan

ExtremeMan Sep 3, 2014

Contributor

Got it. Most likely xctool will not be able to launch simulator as well then.

Contributor

ExtremeMan commented Sep 3, 2014

Got it. Most likely xctool will not be able to launch simulator as well then.

@obuseme

This comment has been minimized.

Show comment
Hide comment
@obuseme

obuseme Sep 4, 2014

I got the exact same problem with xcodebuild as @xfreebird

obuseme commented Sep 4, 2014

I got the exact same problem with xcodebuild as @xfreebird

@eduaranega

This comment has been minimized.

Show comment
Hide comment
@eduaranega

eduaranega Sep 4, 2014

Similar issue has been reported on the developer forum:
https://devforums.apple.com/message/1033457#1033457

eduaranega commented Sep 4, 2014

Similar issue has been reported on the developer forum:
https://devforums.apple.com/message/1033457#1033457

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 10, 2014

Still happens and with the latest Xcode 6 (A313).

xfreebird commented Sep 10, 2014

Still happens and with the latest Xcode 6 (A313).

@SRandazzo

This comment has been minimized.

Show comment
Hide comment
@SRandazzo

SRandazzo Sep 10, 2014

Same issue here, have tried the freshinstall option as well as launching the simulator before tests, no luck

SRandazzo commented Sep 10, 2014

Same issue here, have tried the freshinstall option as well as launching the simulator before tests, no luck

@leafduo

This comment has been minimized.

Show comment
Hide comment
@leafduo

leafduo Sep 12, 2014

Does this issue have a workaround of any kind? Apple seems not going to change this behavior.

leafduo commented Sep 12, 2014

Does this issue have a workaround of any kind? Apple seems not going to change this behavior.

@zygoat

This comment has been minimized.

Show comment
Hide comment
@zygoat

zygoat Sep 12, 2014

This has been under discussion at https://devforums.apple.com/message/1040016#1040016. I opened a bug today (18311014) at the request of an Apple engineering manager who finally chimed in to that thread. Hopefully it will get addressed soon.

zygoat commented Sep 12, 2014

This has been under discussion at https://devforums.apple.com/message/1040016#1040016. I opened a bug today (18311014) at the request of an Apple engineering manager who finally chimed in to that thread. Hopefully it will get addressed soon.

@LegNeato

This comment has been minimized.

Show comment
Hide comment
@LegNeato

LegNeato Sep 19, 2014

Contributor

The new release with Xcode 6 support (v0.2) is now in Homebrew. Please confirm it fixes this issue for you.

Contributor

LegNeato commented Sep 19, 2014

The new release with Xcode 6 support (v0.2) is now in Homebrew. Please confirm it fixes this issue for you.

@zpatten

This comment has been minimized.

Show comment
Hide comment
@zpatten

zpatten Sep 19, 2014

I'm still seeing this issue using v0.2.1 (i.e. HEAD) via brew.

zpatten commented Sep 19, 2014

I'm still seeing this issue using v0.2.1 (i.e. HEAD) via brew.

@ExtremeMan

This comment has been minimized.

Show comment
Hide comment
@ExtremeMan

ExtremeMan Sep 19, 2014

Contributor

I think we can do nothing from xctool perspective.

Contributor

ExtremeMan commented Sep 19, 2014

I think we can do nothing from xctool perspective.

@zpatten

This comment has been minimized.

Show comment
Hide comment
@zpatten

zpatten Sep 19, 2014

Ya I agree; it's all in Apple's hands at this point.

zpatten commented Sep 19, 2014

Ya I agree; it's all in Apple's hands at this point.

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Sep 29, 2014

So, one workaround (not very like-able though) is reportedly to use the Java Web Star agent in Jenkins instead of an SSH agent. Any other known workaround? I'd certainly expect more Jenkins users complaining here... or is everyone moving to Xcode Bots?

oscahie commented Sep 29, 2014

So, one workaround (not very like-able though) is reportedly to use the Java Web Star agent in Jenkins instead of an SSH agent. Any other known workaround? I'd certainly expect more Jenkins users complaining here... or is everyone moving to Xcode Bots?

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 29, 2014

My temporary solution was to start another ssh daemon on a different port from UI session by adding a script to Login Items.

xfreebird commented Sep 29, 2014

My temporary solution was to start another ssh daemon on a different port from UI session by adding a script to Login Items.

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Sep 30, 2014

@xfreebird that would work out for me too as a temp solution. Can you share the script?

oscahie commented Sep 30, 2014

@xfreebird that would work out for me too as a temp solution. Can you share the script?

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Sep 30, 2014

👍 that worked, thank you! I've added my ssh daemon as a user LaunchAgent instead though, cause I had some issues getting the login item executed, plus then there is no need to programmatically hide the terminal window. Anyhow, this workaround should do fine until Apple fixes their tools (if ever...).

oscahie commented Sep 30, 2014

👍 that worked, thank you! I've added my ssh daemon as a user LaunchAgent instead though, cause I had some issues getting the login item executed, plus then there is no need to programmatically hide the terminal window. Anyhow, this workaround should do fine until Apple fixes their tools (if ever...).

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 30, 2014

I did try adding it to be started by LaunchAgent, but I had the same issue. Could you please share your LaunchAgent config ?

xfreebird commented Sep 30, 2014

I did try adding it to be started by LaunchAgent, but I had the same issue. Could you please share your LaunchAgent config ?

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Sep 30, 2014

Sure, it's extremely simple though:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.redacted.customsshd</string>
   <key>Program</key>
   <string>/Users/jenkins/custom_sshd.sh</string>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>

Added it as /Users/jenkins/Library/LaunchAgents/com.redacted.customsshd.plist. After rebooting the machine it still worked, but I'm crossing my fingers...

oscahie commented Sep 30, 2014

Sure, it's extremely simple though:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.redacted.customsshd</string>
   <key>Program</key>
   <string>/Users/jenkins/custom_sshd.sh</string>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>

Added it as /Users/jenkins/Library/LaunchAgents/com.redacted.customsshd.plist. After rebooting the machine it still worked, but I'm crossing my fingers...

@xfreebird

This comment has been minimized.

Show comment
Hide comment
@xfreebird

xfreebird Sep 30, 2014

Thanks ! Indeed it is working. Not sure why it was not working before. I did update the script, now it has an option to install itself as a launchctl agent

xfreebird commented Sep 30, 2014

Thanks ! Indeed it is working. Not sure why it was not working before. I did update the script, now it has an option to install itself as a launchctl agent

@BobReid

This comment has been minimized.

Show comment
Hide comment
@BobReid

BobReid Oct 21, 2014

I am still seeing this issue. I use TeamCity as my build server and the builds are kicked off from a LaunchAgent process. Is there anything else that I should be doing to work around this issue?

BobReid commented Oct 21, 2014

I am still seeing this issue. I use TeamCity as my build server and the builds are kicked off from a LaunchAgent process. Is there anything else that I should be doing to work around this issue?

@joshuafinch

This comment has been minimized.

Show comment
Hide comment
@joshuafinch

joshuafinch Oct 25, 2014

I'm still seeing this issue, using version 0.2.1 from brew. Running xctool works fine when logged in as the user through terminal, but when commands are sent over ssh with the user still logged in, it fails to launch the simulator correctly.

You can visually see the simulator being launched, then relaunched twice without being given enough time to fully launch and install the application for testing.

Have Xcode 6.1 and OSX 10.10 installed, with both iOS 7.1 and 8.1 simulators downloaded.

joshuafinch commented Oct 25, 2014

I'm still seeing this issue, using version 0.2.1 from brew. Running xctool works fine when logged in as the user through terminal, but when commands are sent over ssh with the user still logged in, it fails to launch the simulator correctly.

You can visually see the simulator being launched, then relaunched twice without being given enough time to fully launch and install the application for testing.

Have Xcode 6.1 and OSX 10.10 installed, with both iOS 7.1 and 8.1 simulators downloaded.

@ligerjohn

This comment has been minimized.

Show comment
Hide comment
@ligerjohn

ligerjohn Oct 31, 2014

Seeing the same issue. Running 10.10 / 6.1 / 0.2.1 as well. Pasted the output from a failed xctool run-tests -showTasks below. Works if I'm not using ssh, fails if I am.

My .xctool-args

[
  "-jobs", "4",
  "-reporter", "pretty",
  "-workspace", "AnApp.xcworkspace",
  "-scheme", "AnApp",
  "-sdk", "iphonesimulator8.1",
  "-destination", "name=iPhone 6"
]

This is stopping us from running our continuous integration so I am very willing to help looking for the issue and try things out, though it should be fairly easy to test. Just ssh to localhost and run the test. I also see the simulator starting and then something fails. Worked with Xcode 5.x/iOS7.x.

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (spawning reporter task):

  /usr/local/Cellar/xctool/0.2.1/libexec/reporters/pretty
================================================================================

================================================================================
LAUNCHING TASK (querying available SDKs):

  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -sdk \
    -version
================================================================================

================================================================================
LAUNCHING TASK (gathering build settings for a target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_FIRST_BUILDABLE=YES \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -workspace \
    AnApp.xcworkspace \
    -scheme \
    AnApp \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    build \
    -showBuildSettings
================================================================================
[Info] Loading settings for scheme 'AnApp' ... (1759 ms)

=== RUN-TESTS ===

  [Info] Collecting info for testables...
================================================================================
LAUNCHING TASK (running xcodebuild -showBuildSettings for 'AnAppTests' target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_TARGET=AnAppTests \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -configuration \
    Debug \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    -project \
    AnApp.xcodeproj \
    -target \
    AnAppTests \
    OBJROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates \
    SYMROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products \
    SHARED_PRECOMPS_DIR=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates/PrecompiledHeaders \
    TARGETED_DEVICE_FAMILY=1 \
    test \
    -showBuildSettings
================================================================================

================================================================================
LAUNCHING TASK (running otest-query):

  SIMSHIM_DYLD_FALLBACK_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks \
  SIMSHIM_DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/otest-query-lib-ios.dylib \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/sim-shim.dylib \
  SIMSHIM_OtestQueryBundlePath=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnAppTests.xctest \
  SIMSHIM___CFPREFERENCES_AVOID_DAEMON=YES \
  /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/sim \
    --arch=64 \
    --sdk=8.1 \
    --environment=merge \
    /Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnApp.app/AnApp
================================================================================
  [Info] Collecting info for testables... (1320 ms)
  run-test AnAppTests.xctest (iphonesimulator8.1, iPhone 6, application-test)
    [Info] Installed 'com.AnApp.mobile.ios.AdvertiserApp'. (746 ms)
    [Info] Launching test host and running tests ... (0 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 2 more times. (0 ms)
    [Info] Shut down iOS Simulator... (1120 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 1 more time. (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Warning] The simulator failed to start, or the TEST_HOST application failed to run.. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[TEST_BUNDLE FAILED_TO_START] (0 ms) (0)
      -[NumberUtilsTest testPhoneNumber]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[NumberUtilsTest testPhoneNumber] (0 ms) (1)
    0 passed, 0 failed, 2 errored, 2 total (0 ms)


Failures:

  0) -[TEST_BUNDLE FAILED_TO_START] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  1) -[NumberUtilsTest testPhoneNumber] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

** RUN-TESTS FAILED: 0 passed, 0 failed, 2 errored, 2 total ** (15681 ms)

ligerjohn commented Oct 31, 2014

Seeing the same issue. Running 10.10 / 6.1 / 0.2.1 as well. Pasted the output from a failed xctool run-tests -showTasks below. Works if I'm not using ssh, fails if I am.

My .xctool-args

[
  "-jobs", "4",
  "-reporter", "pretty",
  "-workspace", "AnApp.xcworkspace",
  "-scheme", "AnApp",
  "-sdk", "iphonesimulator8.1",
  "-destination", "name=iPhone 6"
]

This is stopping us from running our continuous integration so I am very willing to help looking for the issue and try things out, though it should be fairly easy to test. Just ssh to localhost and run the test. I also see the simulator starting and then something fails. Worked with Xcode 5.x/iOS7.x.

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (spawning reporter task):

  /usr/local/Cellar/xctool/0.2.1/libexec/reporters/pretty
================================================================================

================================================================================
LAUNCHING TASK (querying available SDKs):

  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -sdk \
    -version
================================================================================

================================================================================
LAUNCHING TASK (gathering build settings for a target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_FIRST_BUILDABLE=YES \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -workspace \
    AnApp.xcworkspace \
    -scheme \
    AnApp \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    build \
    -showBuildSettings
================================================================================
[Info] Loading settings for scheme 'AnApp' ... (1759 ms)

=== RUN-TESTS ===

  [Info] Collecting info for testables...
================================================================================
LAUNCHING TASK (running xcodebuild -showBuildSettings for 'AnAppTests' target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_TARGET=AnAppTests \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -configuration \
    Debug \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    -project \
    AnApp.xcodeproj \
    -target \
    AnAppTests \
    OBJROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates \
    SYMROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products \
    SHARED_PRECOMPS_DIR=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates/PrecompiledHeaders \
    TARGETED_DEVICE_FAMILY=1 \
    test \
    -showBuildSettings
================================================================================

================================================================================
LAUNCHING TASK (running otest-query):

  SIMSHIM_DYLD_FALLBACK_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks \
  SIMSHIM_DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/otest-query-lib-ios.dylib \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/sim-shim.dylib \
  SIMSHIM_OtestQueryBundlePath=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnAppTests.xctest \
  SIMSHIM___CFPREFERENCES_AVOID_DAEMON=YES \
  /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/sim \
    --arch=64 \
    --sdk=8.1 \
    --environment=merge \
    /Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnApp.app/AnApp
================================================================================
  [Info] Collecting info for testables... (1320 ms)
  run-test AnAppTests.xctest (iphonesimulator8.1, iPhone 6, application-test)
    [Info] Installed 'com.AnApp.mobile.ios.AdvertiserApp'. (746 ms)
    [Info] Launching test host and running tests ... (0 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 2 more times. (0 ms)
    [Info] Shut down iOS Simulator... (1120 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 1 more time. (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Warning] The simulator failed to start, or the TEST_HOST application failed to run.. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[TEST_BUNDLE FAILED_TO_START] (0 ms) (0)
      -[NumberUtilsTest testPhoneNumber]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[NumberUtilsTest testPhoneNumber] (0 ms) (1)
    0 passed, 0 failed, 2 errored, 2 total (0 ms)


Failures:

  0) -[TEST_BUNDLE FAILED_TO_START] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  1) -[NumberUtilsTest testPhoneNumber] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

** RUN-TESTS FAILED: 0 passed, 0 failed, 2 errored, 2 total ** (15681 ms)
@scottknight

This comment has been minimized.

Show comment
Hide comment
@scottknight

scottknight Nov 6, 2014

@BobReid did you ever get things working with TeamCity? I'm also using TeamCity and not sure how to get things going since the LaunchAgent runs things.

scottknight commented Nov 6, 2014

@BobReid did you ever get things working with TeamCity? I'm also using TeamCity and not sure how to get things going since the LaunchAgent runs things.

@perry-mitchell

This comment has been minimized.

Show comment
Hide comment
@perry-mitchell

perry-mitchell Nov 7, 2014

Still seeing this issue over SSH with the latest commit ver in xctool github: 106cb80

Running: xctool.sh -scheme <scheme> -sdk iphonesimulator -workspace <proj>/project.xcworkspace -destination 'name=iPhone 6,OS=8.1' test

And getting:

  run-test <name>Tests.xctest (8.1, iPhone 6, application-test)
    [Warning] Tried to install the test host app '<id>' but failed. (53 ms)
    [Info] Preparing test environment failed; will retry 2 more times (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (124 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (1088 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Info] Preparing test environment failed; will retry 1 more time (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (133 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (496 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Warning] Preparing test environment failed. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: Simulator 'iPhone 6' was not prepared: Failed for unknown reason.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

perry-mitchell commented Nov 7, 2014

Still seeing this issue over SSH with the latest commit ver in xctool github: 106cb80

Running: xctool.sh -scheme <scheme> -sdk iphonesimulator -workspace <proj>/project.xcworkspace -destination 'name=iPhone 6,OS=8.1' test

And getting:

  run-test <name>Tests.xctest (8.1, iPhone 6, application-test)
    [Warning] Tried to install the test host app '<id>' but failed. (53 ms)
    [Info] Preparing test environment failed; will retry 2 more times (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (124 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (1088 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Info] Preparing test environment failed; will retry 1 more time (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (133 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (496 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Warning] Preparing test environment failed. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: Simulator 'iPhone 6' was not prepared: Failed for unknown reason.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

@binl

This comment has been minimized.

Show comment
Hide comment
@binl

binl Nov 15, 2014

@xfreebird @oscahie Guys! ur solution did help me launch the simulator, but I still got the app failed to install, any thoughts?

binl commented Nov 15, 2014

@xfreebird @oscahie Guys! ur solution did help me launch the simulator, but I still got the app failed to install, any thoughts?

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Nov 15, 2014

I've personally given up on xctool, for now at least. After updating Xcode to 6.1, xctool was failing somewhere else (can't remember if it was the app installation or what). Anyway, I've switched back to xcodebuild, it's just less problematic (still need the user-launched ssh daemon though) plus I can now run the test cases on actual iOS devices (haven't done it yet but planning to).

oscahie commented Nov 15, 2014

I've personally given up on xctool, for now at least. After updating Xcode to 6.1, xctool was failing somewhere else (can't remember if it was the app installation or what). Anyway, I've switched back to xcodebuild, it's just less problematic (still need the user-launched ssh daemon though) plus I can now run the test cases on actual iOS devices (haven't done it yet but planning to).

@ligerjohn

This comment has been minimized.

Show comment
Hide comment
@ligerjohn

ligerjohn Nov 15, 2014

For reference, xcodebuild also fails to work inside of ssh.

xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination 'OS=8.1,name=iPhone 6' test

Gives us the following epic fail:

2014-11-14 20:52:29.868 Beginning test session with Xcode 6A1052d
2014-11-14 20:52:29.868 /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination OS=8.1,name=iPhone 6 test
2014-11-14 20:52:29.868 Testing on device: <DVTiPhoneSimulator: 0x7feb6c3379c0> {
                SimDevice: SimDevice : iPhone 6 (37D19143-24DD-4E77-9E1D-359066F7F38F) : state={ Shutdown } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-6 } runtime={ SimRuntime : 8.1 (12B411) - com.apple.CoreSimulator.SimRuntime.iOS-8-1 }
}
2014-11-14 20:52:29.868 Initializing test infrastructure.
2014-11-14 20:52:29.868 Writing testing status log to /var/folders/df/yyfqwnv92t36sh91b19pf8h00000gp/T/com.apple.dt.XCTest-status/Session-2014-11-14_20:52:29-rxlV4K.log.
2014-11-14 20:52:34.215 Launch session started, setting a disallow-finish-token on the run operation.
2014-11-14 20:52:34.217 Adding console adaptor for test process.
2014-11-14 20:54:29.872 Test operation failure: Timed out waiting 120 seconds for simulator to boot, current state is 1.
2014-11-14 20:54:29.873 _finishWithError:Error Domain=IDEUnitTestsOperationsObserverErrorDomain Code=3 "Timed out waiting 120 seconds for simulator to boot, current state is 1." UserInfo=0x7feb6dacd090 {NSLocalizedDescription=Timed out waiting 120 seconds for simulator to boot, current state is 1.} didCancel: 1

ligerjohn commented Nov 15, 2014

For reference, xcodebuild also fails to work inside of ssh.

xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination 'OS=8.1,name=iPhone 6' test

Gives us the following epic fail:

2014-11-14 20:52:29.868 Beginning test session with Xcode 6A1052d
2014-11-14 20:52:29.868 /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination OS=8.1,name=iPhone 6 test
2014-11-14 20:52:29.868 Testing on device: <DVTiPhoneSimulator: 0x7feb6c3379c0> {
                SimDevice: SimDevice : iPhone 6 (37D19143-24DD-4E77-9E1D-359066F7F38F) : state={ Shutdown } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-6 } runtime={ SimRuntime : 8.1 (12B411) - com.apple.CoreSimulator.SimRuntime.iOS-8-1 }
}
2014-11-14 20:52:29.868 Initializing test infrastructure.
2014-11-14 20:52:29.868 Writing testing status log to /var/folders/df/yyfqwnv92t36sh91b19pf8h00000gp/T/com.apple.dt.XCTest-status/Session-2014-11-14_20:52:29-rxlV4K.log.
2014-11-14 20:52:34.215 Launch session started, setting a disallow-finish-token on the run operation.
2014-11-14 20:52:34.217 Adding console adaptor for test process.
2014-11-14 20:54:29.872 Test operation failure: Timed out waiting 120 seconds for simulator to boot, current state is 1.
2014-11-14 20:54:29.873 _finishWithError:Error Domain=IDEUnitTestsOperationsObserverErrorDomain Code=3 "Timed out waiting 120 seconds for simulator to boot, current state is 1." UserInfo=0x7feb6dacd090 {NSLocalizedDescription=Timed out waiting 120 seconds for simulator to boot, current state is 1.} didCancel: 1
@benjamin-s-webcenter

This comment has been minimized.

Show comment
Hide comment
@benjamin-s-webcenter

benjamin-s-webcenter Nov 17, 2014

Same issue with XCode 6.1.1 GM

benjamin-s-webcenter commented Nov 17, 2014

Same issue with XCode 6.1.1 GM

@leafduo

This comment has been minimized.

Show comment
Hide comment
@leafduo

leafduo Nov 17, 2014

@xfreebird @oscahie I created a custom LaunchAgent-based sshd daemon, but still cannot launch the simulator, but launching JNLP manually works (not working from launchctl).

I'm curious if it's the version of OS X makes the difference, I'm still using Mavericks on my Jenkins slave. Have you upgraded to Yosemite?

leafduo commented Nov 17, 2014

@xfreebird @oscahie I created a custom LaunchAgent-based sshd daemon, but still cannot launch the simulator, but launching JNLP manually works (not working from launchctl).

I'm curious if it's the version of OS X makes the difference, I'm still using Mavericks on my Jenkins slave. Have you upgraded to Yosemite?

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Nov 17, 2014

We're running 10.9.5 on our CI server.

oscahie commented Nov 17, 2014

We're running 10.9.5 on our CI server.

@kaeawc

This comment has been minimized.

Show comment
Hide comment
@kaeawc

kaeawc Nov 17, 2014

@oscahie The issue is not with xctool, xcodebuild (Xcode 6.1, Build version 6A1052d) cannot properly attach to the iOS Simulator over an SSH session. For now the workaround is to use a manually started JNLP session whether the OSX machine you're attempting to build on is Mavericks (10.9.5) or Yosemite (10.10).

kaeawc commented Nov 17, 2014

@oscahie The issue is not with xctool, xcodebuild (Xcode 6.1, Build version 6A1052d) cannot properly attach to the iOS Simulator over an SSH session. For now the workaround is to use a manually started JNLP session whether the OSX machine you're attempting to build on is Mavericks (10.9.5) or Yosemite (10.10).

@oscahie

This comment has been minimized.

Show comment
Hide comment
@oscahie

oscahie Nov 17, 2014

@kaeawc I'm aware of that. It can also work over an ssh session though if you start your own daemon from the logged-in user, as explained a couple of comments above. This way xcodebuild works well with Xcode 6.1, whereas xctool doesn't (at least it didn't last time I tried, things might have changed by now).

oscahie commented Nov 17, 2014

@kaeawc I'm aware of that. It can also work over an ssh session though if you start your own daemon from the logged-in user, as explained a couple of comments above. This way xcodebuild works well with Xcode 6.1, whereas xctool doesn't (at least it didn't last time I tried, things might have changed by now).

@killian90

This comment has been minimized.

Show comment
Hide comment
@killian90

killian90 Nov 27, 2014

@kaeawc I'm trying to set up the JNLP session to get xctool to work, but I'm getting permission errors -

Building remotely on iOS-Slave in workspace /workspace/Test Project
java.io.IOException: Failed to mkdirs: /workspace/Test Project

Would it be possible for you to walk through your setup or link to any resources you used? Cheers

killian90 commented Nov 27, 2014

@kaeawc I'm trying to set up the JNLP session to get xctool to work, but I'm getting permission errors -

Building remotely on iOS-Slave in workspace /workspace/Test Project
java.io.IOException: Failed to mkdirs: /workspace/Test Project

Would it be possible for you to walk through your setup or link to any resources you used? Cheers

@BobReid

This comment has been minimized.

Show comment
Hide comment
@BobReid

BobReid Nov 27, 2014

@scottknight The LaunchAgent used to work prior to XCode 6. LaunchAgent processes have GUI sessions so I am not sure what the actual problem is with xctool.

I did create a temporary patchwork solution until it gets figured out or we switch build tools. I wrote a small apple script that launches terminal and executes xctool from the terminal window and that works.
Still have a few edge cases where failures are reported as successes that I need to capture but for 99% of build it succeed / fails properly.

BobReid commented Nov 27, 2014

@scottknight The LaunchAgent used to work prior to XCode 6. LaunchAgent processes have GUI sessions so I am not sure what the actual problem is with xctool.

I did create a temporary patchwork solution until it gets figured out or we switch build tools. I wrote a small apple script that launches terminal and executes xctool from the terminal window and that works.
Still have a few edge cases where failures are reported as successes that I need to capture but for 99% of build it succeed / fails properly.

@postmechanical

This comment has been minimized.

Show comment
Hide comment
@postmechanical

postmechanical Dec 16, 2014

Just ran into this issue. OS X 10.10.1 14B25. Xcode 6.1.1 6A2008a. Jenkins 1.594. Not using a slave config at all. Jenkins configured as a launch agent. iOS simulator will not launch for tests via xctool or xcodebuild. Even an AppleScript that tries to launch Terminal app to run the test shell script doesn't work.

It was working prior to upgrading the build system from OS X 10.9.5.

postmechanical commented Dec 16, 2014

Just ran into this issue. OS X 10.10.1 14B25. Xcode 6.1.1 6A2008a. Jenkins 1.594. Not using a slave config at all. Jenkins configured as a launch agent. iOS simulator will not launch for tests via xctool or xcodebuild. Even an AppleScript that tries to launch Terminal app to run the test shell script doesn't work.

It was working prior to upgrading the build system from OS X 10.9.5.

@jollychang

This comment has been minimized.

Show comment
Hide comment
@jollychang

jollychang Dec 17, 2014

Works for me on jenkins JNLP:

  1. on OSX slave,connect with screen share or use Displays
  2. in OSX slave terminal, Lanuch Jenkins JNLP

If connecting with JNLP still does not solve it, try the solution mentioned in this comment.
i.e.: Run these on command line:

DevToolsSecurity -enable
sudo dscl . -append /Groups/_developer GroupMembership jenkins
security authorizationdb write system.privilege.taskport is-developer

Thanks Michael 's answer

Apple follow up regarding Bug ID# 18001199:

The context provided by LaunchDaemons is not supported for running GUI applications. The SSH service, and the default setup for Jenkins, are both implemented as LaunchDaemons. In earlier versions of Xcode 5 xcodebuild could run tests on the iOS simulator in this context, but that was never a supported configuration, and as you have noted that is no longer working as of Xcode 6.

Unlike LaunchDaemons, LaunchAgents provide a context where you can run GUI applications - if the user is logged in at the time, with a window server / Aqua session. Converting your Jenkins configuration from being a LaunchDaemon to being a LaunchAgent would avoid the reported issue. You can also use launchd for running tests on the iOS simulator from a SSH session, either by crafting a LaunchAgent and manually loading / starting that, or by using "launchctl submit”.

jollychang commented Dec 17, 2014

Works for me on jenkins JNLP:

  1. on OSX slave,connect with screen share or use Displays
  2. in OSX slave terminal, Lanuch Jenkins JNLP

If connecting with JNLP still does not solve it, try the solution mentioned in this comment.
i.e.: Run these on command line:

DevToolsSecurity -enable
sudo dscl . -append /Groups/_developer GroupMembership jenkins
security authorizationdb write system.privilege.taskport is-developer

Thanks Michael 's answer

Apple follow up regarding Bug ID# 18001199:

The context provided by LaunchDaemons is not supported for running GUI applications. The SSH service, and the default setup for Jenkins, are both implemented as LaunchDaemons. In earlier versions of Xcode 5 xcodebuild could run tests on the iOS simulator in this context, but that was never a supported configuration, and as you have noted that is no longer working as of Xcode 6.

Unlike LaunchDaemons, LaunchAgents provide a context where you can run GUI applications - if the user is logged in at the time, with a window server / Aqua session. Converting your Jenkins configuration from being a LaunchDaemon to being a LaunchAgent would avoid the reported issue. You can also use launchd for running tests on the iOS simulator from a SSH session, either by crafting a LaunchAgent and manually loading / starting that, or by using "launchctl submit”.

@mrooney

This comment has been minimized.

Show comment
Hide comment
@mrooney

mrooney Mar 9, 2015

@BobReid, would you mind sharing that launch script that launches Web Start via a Terminal?

mrooney commented Mar 9, 2015

@BobReid, would you mind sharing that launch script that launches Web Start via a Terminal?

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 5, 2015

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

ghost commented Aug 5, 2015

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

@ExtremeMan

This comment has been minimized.

Show comment
Hide comment
@ExtremeMan

ExtremeMan Aug 5, 2015

Contributor

Since it is more an Apple issue rather then xctool (and I doubt we can improve/fix it in xctool itself) I will close the issue. Hope you all found a workaround.

Feel free to send a PR with your workarounds. We can add all of them to a README, I guess, to help other engineers when they hit this issue.

Contributor

ExtremeMan commented Aug 5, 2015

Since it is more an Apple issue rather then xctool (and I doubt we can improve/fix it in xctool itself) I will close the issue. Hope you all found a workaround.

Feel free to send a PR with your workarounds. We can add all of them to a README, I guess, to help other engineers when they hit this issue.

@ExtremeMan ExtremeMan closed this Aug 5, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment