XCTest Hangs when Used with Scan on Bitrise with Xcode 8 #6111

Closed
Ben-G opened this Issue Sep 14, 2016 · 12 comments

Comments

Projects
None yet
7 participants
@Ben-G

Ben-G commented Sep 14, 2016

New Issue Checklist

Issue Description

When running scan via fastlane on Bitrise with Xcode 8, the unit tests will always hang and fail the build. This issue does not occur on Bitrise when calling xcodebuild directly. The issue does also not occur when running fastlane on a local machine - I could only reproduce the issue in this combination. I'm wondering if others are seeing similar issues on other CI providers?

Fastlane fails with exit code 65 when attempting to run unit tests, without any additional information:

▸ Running script '[CP] Copy Pods Resources'
▸ Touching PlanGridUnitTests.xctest
▸ Signing /Users/vagrant/Library/Developer/Xcode/DerivedData/PlanGrid-fwcdtyfkquxsxabgkthlycyyhpvt/Build/Intermediates/CodeCoverage/Products/Debug-iphonesimulator/PlanGrid.app/PlugIns/PlanGridUnitTests.xctest
** TEST FAILED **
[14:59:19]: Exit status: 65
[14:59:20]: Variable Dump:
[14:59:20]: {:DEFAULT_PLATFORM=>:ios, :PLATFORM_NAME=>:ios, :LANE_NAME=>"ios dev", :SCAN_DERIVED_DATA_PATH=>"/Users/vagrant/Library/Developer/Xcode/DerivedData/PlanGrid-fwcdtyfkquxsxabgkthlycyyhpvt", :SCAN_GENERATED_PLIST_FILES=>[], :SCAN_GENERATED_PLIST_FILE=>nil}
[14:59:20]: Error building/testing the application - see the log above

The xcactivitylog from the test run is looks as following and provides more details:

14:50:25.982 xcodebuild[1706:14490] Returning /private/tmp/com.apple.launchd.O2dWsgK0Ma/com.apple.testmanagerd.unix-domain.socket for TESTMANAGERD_SIM_SOCK
14:50:25.982 xcodebuild[1706:14490] Sim iPad Air 2 has testmanagerd socket at /private/tmp/com.apple.launchd.O2dWsgK0Ma/com.apple.testmanagerd.unix-domain.socket
14:50:25.982 xcodebuild[1706:14490] Connected socket 32 to testmanagerd for Sim iPad Air 2
14:50:25.988 xcodebuild[1706:8431] Returning /private/tmp/com.apple.launchd.O2dWsgK0Ma/com.apple.testmanagerd.unix-domain.socket for TESTMANAGERD_SIM_SOCK
14:50:25.988 xcodebuild[1706:8431] Sim iPad Air 2 has testmanagerd socket at /private/tmp/com.apple.launchd.O2dWsgK0Ma/com.apple.testmanagerd.unix-domain.socket
14:50:25.988 xcodebuild[1706:8431] Connected socket 35 to testmanagerd for Sim iPad Air 2
14:50:26.016 xcodebuild[1706:8431] Creating the test bundle connection.
14:50:26.043 xcodebuild[1706:8431] Listening for proxy connection request from the test bundle (all platforms)
14:50:26.043 xcodebuild[1706:8431] Resuming the connection.
14:50:26.043 xcodebuild[1706:8431] Test connection requires daemon assistance.
14:50:26.859 xcodebuild[1706:8363] Starting test session with ID <__NSConcreteUUID 0x7fe44fc7d580> BBFEE52F-8CCB-42F7-B095-099301FE730A
14:50:26.859 xcodebuild[1706:8363] Checking test manager availability..., will wait up to 120s
14:52:26.863 xcodebuild[1706:8363] Unable to get debug console for logging target process thread state.
14:52:26.865 xcodebuild[1706:8363] Test operation failure: Canceling tests due to timeout in Checking test manager availability...
14:52:26.865 xcodebuild[1706:8363] _finishWithError:Error Domain=IDETestOperationsObserverErrorDomain Code=3 "Canceling tests due to timeout in Checking test manager availability..." UserInfo={NSLocalizedDescription=Canceling tests due to timeout in Checking test manager availability...} didCancel: 1

The full log can be found here.

Configuration Files

Relevant Fastfile lane:

  desc "Runs all the tests & uploads code coverage data to codecov.io and updates translation keys on Phraseapp"
  lane :dev do
    # Upload latest localization keys to Phraseapp
    UpdateLocalization.upload_latest_localization

    scan(
      scheme: "PlanGrid_CI",
      devices: ["iPhone SE"],
      clean: false,
      code_coverage: true
    )

    sh "../scripts/upload_codecoverage.sh"
  end

Environment

fastlane version (run fastlane -v):

Do you use bundler to execute fastlane (i.e. bundle exec fastlane)?

Yes

Do you use a Ruby environment manager (e.g. chruby, rbenv, rvm)?

No, bitrise uses homebrew's ruby version. Currently 2.3.1.

@fastlane-bot

This comment has been minimized.

Show comment
Hide comment
@fastlane-bot

fastlane-bot Sep 14, 2016

It seems like this issue might be related to code signing 🚫

Have you seen our new Code Signing Troubleshooting Guide? It will help you resolve the most common code signing issues 👍

It seems like this issue might be related to code signing 🚫

Have you seen our new Code Signing Troubleshooting Guide? It will help you resolve the most common code signing issues 👍

@Ben-G

This comment has been minimized.

Show comment
Hide comment
@Ben-G

Ben-G Sep 14, 2016

Not related to code signing - but good guess @fastlane-bot!

Ben-G commented Sep 14, 2016

Not related to code signing - but good guess @fastlane-bot!

@TKBurner

This comment has been minimized.

Show comment
Hide comment
@TKBurner

TKBurner Sep 14, 2016

@Ben-G Hope you are doing well! Does this happen in Xcode 8 locally as well or just on Bitrise? Can you share your full input and output? Thanks!

@Ben-G Hope you are doing well! Does this happen in Xcode 8 locally as well or just on Bitrise? Can you share your full input and output? Thanks!

@Ben-G

This comment has been minimized.

Show comment
Hide comment
@Ben-G

Ben-G Sep 14, 2016

@TKBurner Thanks for getting back on this. This only occurs on bitrise. Running on a local machine it works fine. However, I can run tests on bitrise when only using xcodebuild and not using fastlane. So this appears to be an issue that only occurs when both tools are combined :|

As for input/output: you can find the full log file of the build here. I shared the Fastfile here.

Ben-G commented Sep 14, 2016

@TKBurner Thanks for getting back on this. This only occurs on bitrise. Running on a local machine it works fine. However, I can run tests on bitrise when only using xcodebuild and not using fastlane. So this appears to be an issue that only occurs when both tools are combined :|

As for input/output: you can find the full log file of the build here. I shared the Fastfile here.

@TKBurner

This comment has been minimized.

Show comment
Hide comment
@TKBurner

TKBurner Sep 22, 2016

Hi @Ben-G So sorry for the delay in getting back to you. Can you let me know what happens if you set

clean: true

or erase_simulators prior to running your lane. Thanks and hope you have a great week!

Hi @Ben-G So sorry for the delay in getting back to you. Can you let me know what happens if you set

clean: true

or erase_simulators prior to running your lane. Thanks and hope you have a great week!

@scottrhoyt

This comment has been minimized.

Show comment
Hide comment
@scottrhoyt

scottrhoyt Sep 24, 2016

Contributor

Same thing for me on travis and Xcode 8. Can't find an erase_simulators action. Is that perhaps reset_simulators now?

Contributor

scottrhoyt commented Sep 24, 2016

Same thing for me on travis and Xcode 8. Can't find an erase_simulators action. Is that perhaps reset_simulators now?

@scottrhoyt

This comment has been minimized.

Show comment
Hide comment
@scottrhoyt

scottrhoyt Sep 24, 2016

Contributor

FWIW clean: true and running reset_simulators did not work for me.

Contributor

scottrhoyt commented Sep 24, 2016

FWIW clean: true and running reset_simulators did not work for me.

@TKBurner

This comment has been minimized.

Show comment
Hide comment
@TKBurner

TKBurner Oct 4, 2016

@scottrhoyt Can you share your full input and output so I can take a look? Thanks so much!

TKBurner commented Oct 4, 2016

@scottrhoyt Can you share your full input and output so I can take a look? Thanks so much!

@carsten-wenderdel

This comment has been minimized.

Show comment
Hide comment
@carsten-wenderdel

carsten-wenderdel Oct 7, 2016

Hi everybody, I've had the same problem with the Xcode 8.0 stack on bitrise. (fastlane/scan on my computer and xcodebuild on bitrise also worked.)
Using the Xcode 8.1 stack on bitrise has solved the problem for me - now fastlane/scan works there as well! I'm not sure whether this due to the newer Xcode or newer fastlane on bitrise.

carsten-wenderdel commented Oct 7, 2016

Hi everybody, I've had the same problem with the Xcode 8.0 stack on bitrise. (fastlane/scan on my computer and xcodebuild on bitrise also worked.)
Using the Xcode 8.1 stack on bitrise has solved the problem for me - now fastlane/scan works there as well! I'm not sure whether this due to the newer Xcode or newer fastlane on bitrise.

@Ben-G

This comment has been minimized.

Show comment
Hide comment
@Ben-G

Ben-G Oct 10, 2016

@carsten-wenderdel Thanks a lot for letting me know about this. It appears the issue is indeed fixed with the Xcode 8.1 stack. 🎉

Closing this issue.

Ben-G commented Oct 10, 2016

@carsten-wenderdel Thanks a lot for letting me know about this. It appears the issue is indeed fixed with the Xcode 8.1 stack. 🎉

Closing this issue.

@Ben-G Ben-G closed this Oct 10, 2016

@KrauseFx

This comment has been minimized.

Show comment
Hide comment
@KrauseFx

KrauseFx Oct 10, 2016

Member

Great to hear this is resolved now, thanks for updating this issue 👍

Member

KrauseFx commented Oct 10, 2016

Great to hear this is resolved now, thanks for updating this issue 👍

@werner77

This comment has been minimized.

Show comment
Hide comment
@werner77

werner77 Nov 24, 2016

See my solution here: http://stackoverflow.com/a/40790171/480467

The problem lies in the fact that the timeout counter in xcodebuild starts as soon as the command is executed, so the compile time counts towards the timeout (which is 120 seconds)

See my solution here: http://stackoverflow.com/a/40790171/480467

The problem lies in the fact that the timeout counter in xcodebuild starts as soon as the command is executed, so the compile time counts towards the timeout (which is 120 seconds)

@fastlane fastlane locked and limited conversation to collaborators Feb 22, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.