"Failed to run tests" but no helpful information given

[adamyanalunas]

I'm using issue #16 as a guide to get my project (which uses CocoaPods and Kiwi) to have tests running on the command line so I can have CI useful.

Running with the following command does all of the pre-test test successfully but fails when attempting to run the app in the simulator.

xctool -workspace MyApp.xcworkspace -scheme MyApp ONLY_ACTIVE_ARCH=NO -configuration Debug test

Where I can I look for reasons why the test suite failed to run?

[fpotter]

Hrm - can you paste in the output for us?

[adamyanalunas]

@fpotter Whoops, I should have pasted that in. Here you go:

$ xctool -workspace MyApp.xcworkspace -scheme MyApp ONLY_ACTIVE_ARCH=NO test
[Info] Collecting build settings ... (1282 ms)

=== TEST ===

  xcodebuild build build
    Pods / Pods (Debug)
      ✓ Check dependencies (323 ms)

    Pods / Pods-KiwiUnitTest (Debug)
      ✓ Check dependencies (157 ms)

    MyApp / MyApp (Debug)
      ✓ Check dependencies (303 ms)
      ✓ Run custom shell script 'Copy Pods Resources' (67 ms)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
rsync -av --exclude '*/.svn/*' /Users/thatguy/Code/MyApp/Pods/ZAActivityBar/ZAActivityBar/ZAActivityBar.bundle /Users/thatguy/Library/Developer/Xcode/DerivedData/MyApp-fvyajjdocblkjhfmmzrsrkgbrqgh/Build/Products/Debug-iphonesimulator/MyApp.app
building file list ... done

sent 157 bytes  received 20 bytes  354.00 bytes/sec
total size is 7055  speedup is 39.86
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

    MyApp / KiwiUnitTest (Debug)
      ✓ Check dependencies (10 ms)
      ✓ Run custom shell script 'Run Script' (19 ms)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
/Applications/Xcode.app/Contents/Developer/Tools/RunUnitTests:68: note: RunUnitTests exited without running tests because TEST_AFTER_BUILD was set to NO.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
      ✓ Run custom shell script 'Copy Pods Resources' (17 ms)


  run-test KiwiUnitTest.octest (iphonesimulator6.1, application-test, GC OFF)
    [Info] Installing '/Users/thatguy/Library/Developer/Xcode/DerivedData/MyApp-fvyajjdocblkjhfmmzrsrkgbrqgh/Build/Products/Debug    [Info] Installing '/Users/thatguy/Library/Developer/Xcode/DerivedData/MyApp-fvyajjdocblkjhfmmzrsrkgbrqgh/Build/Products/Debug-iphonesimulator/MyApp.app' ... (1215 ms)
    [Info] Launching test host and running tests ...
  failed: Failed to run tests
** TEST FAILED: 0 of 0 tests passed ** (5307 ms)

Of note I'm using the HEAD version of xctool from Homebrew. I've also checked on permissions for various files and folders; everything looks good.

[jdeff]

I'm running into this issue as well. It seems to happen if I add MessageUI.framework to my target. When I remove that library, the tests run just fine.

Xcode: Version 4.6.3 (4H1503)
OS: 10.8.4
Xctool: 0.1.7

[jdeff]

So my issue was that my application was crashing when it was launched. Adding MessageUI.framework is what exposed it. The crash ended up being because of a class name collision.

You can easily watch the OS console while running xctool to see if your app crashes. However, I do hope that xctool can identify this and log something more helpful to the user. Something like even as simple as

 run-test KiwiUnitTest.octest (iphonesimulator6.1, application-test, GC OFF)
    [Info] Installing '/Users/thatguy/Library/Developer/Xcode/DerivedData/MyApp-fvyajjdocblkjhfmmzrsrkgbrqgh/Build/Products/Debug    [Info] Installing '/Users/thatguy/Library/Developer/Xcode/DerivedData/MyApp-fvyajjdocblkjhfmmzrsrkgbrqgh/Build/Products/Debug-iphonesimulator/MyApp.app' ... (1215 ms)
    [Info] Launching test host and running tests ...
    [Error] Test host has crashed

[fpotter]

0.1.14 now has much better reporting when application tests fail to run. If the tests never start, xctool should give the the stdout/stderr from the test host so you know what's going on.