29033348: Test Coverage reports from xcodebuild CLI ignore coverage from UI tests

[openradar-mirror]

Description

Summary:
A project with both Unit Tests and UI Tests will generate code coverage when ran in the XCode IDE. If the same project is run using xcodebuild test from CLI, code coverage from the Unit Tests will exist but coverage from UI Tests will be missing

Sample project with UI Tests and Unit Tests, showing a difference in coverage from xcrun llvm-cov report between the project built via IDE and via CLI

https://github.com/iwllyu/UITestCoverageBug

thelvis4 found a discrepancy between how the two building processes differ - fastlane-community/xcov#22 (comment)

Steps to Reproduce:

1. install XCode 8.1 with Swift 3
2. clone the UITestCoverageBug project from https://github.com/iwllyu/UITestCoverageBug
3. Open the project in XCode and run the tests. See both Unit and UI tests get run. See both testedViaUnitTest and testedViaUITest are covered
4. in terminal, run xcrun llvm-cov report (see github for details)
5. check that coverage for ViewController.swift is 80%
6. in terminal, run xcodebuild test for project (see github for details)
7. in terminal, run xcrun llvm-cov report (see github for details)
8. check that coverage for ViewController.swift is now 40%

Expected Results:
coverage for ViewController.swift is 80%

Actual Results:
coverage for ViewController.swift is now 40%

Version:
XCode 8.1
OSX 10.11.5

Notes:

Configuration:

Attachments:

Product Version:
Created: 2016-10-31T22:04:11.002410
Originated: 2016-10-31T00:00:00
Open Radar Link: http://www.openradar.me/29033348