New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Always 0% #22
Comments
So I found a difference between using xcodebuild and the IDE So I get a correctly generated report if I run the test with the Xcode IDE, but if I use xcodebuild it produces nothing, what params need to be passed in with xcodebuild? My current command is as follows Update:
|
Hi @Streel88 👋 Try adding the |
Hi @nakiostudio I tried that and still no luck, to see if you can replicate my issue
You should see the output as 0% Note: I am seeing the exact same behaviour with Slather Update:
|
Hi again @Streel88 👋 You are absolutely right, the |
Still no luck all that does is change the build configuration, from what I've learned about it is one of the executables it produces under ../CodeCoverage/.. needs to be executed which it doesn't seem to do with xcodebuild. I will keep looking into it further to see if I can't figure out the correct command |
Honestly, I have no idea what's wrong... if you can publish a sample project with this problem so I can clone and test it. I haven't been able to repro this on my machine. |
Sorry for the delayed reply, here is the dummy Project I put together to test Scenarios
Commands |
I have the same issue. Is there any updated information related with this? |
I have the same issue with my UI tests, when I run |
After some investigations, here is what I've noticed:
Comparing Xcode:
xcodebuild:
So, when running tests using
Running Unit tests with P.S. If we could figure out how to run UI Tests using |
Great stuff @thelvis4 👍 I've been digging a bit into it and there is very little information. I will test a couple of things during the weekend though. Thanks! |
@thelvis4,@nakiostudio thanks for your great effort |
@thelvis4 have you filed a radar? Just wanting to dupe it if you have one already :) |
@raven Sorry, haven't had the time to further investigate (with Xcode 8) and to file a radar. But I would be glad to dupe the radar if you could file one. I believe we should check this on Xcode 8, as I doubt they will release another version of Xcode 7. |
At Codecov we have resolved this issue with the help of customers. It appears that Hope this is related and find it useful. Thank you! |
Hi @stevepeak 👋 Thanks for the heads up, however the issue we are facing is pretty different. xcov processes swift 3 coverage reports without any problems. In addition, xcov works directly with Thank you once again and keep up the good work with Codecov! |
I created a project that demonstrates this bug https://github.com/iwllyu/UITestCoverageBug If you run the tests via XCode IDE you get 80% of ViewController.swift coverage. If you run the tests via xcodebuild CLI you only get 40% coverage for ViewController.swift. This was done with Xcode 7.3.1 - but I read that it's the same for XCode 8. I can test tonight when I get on my home machine. I'll file a radar if I confirm it on XCode 8 I filed a radar and also submitted to open rader here - https://openradar.appspot.com/29033348 not sure what you guys mean by "dupe", hopefully this is what you meant! |
@iwllyu Thank you for submitting the radar and putting it on open radar as well. 👍 |
Thank you @iwllyu! I will take a look this weekend. Hopefully we will come up with a final solution for this issue. |
Has a solution been found? |
I got this on on 12/16/2016 @ 3:17AM
|
@iwllyu thank you for helping me retain my sanity! |
@EliseLeb This is definitely an inconsistency (bug) in Apple's tools. And it's still reproducible with Xcode 8.2. Xcode IDE and
I couldn't make (force) At this point, I will wait for Apple to fix this bug/inconsistency. BTW, my radar was also marked as |
@thelvis4 Thanks for the update! I also filed a bug with apple but not holding my breath. What a bummer for UI tests. |
I've filed a radar too - thanks everyone in this thread for saving me a few extra hours working out what was going on! Has anyone tried in the latest beta yet? |
@neonichu told me that the ticket is under investigation. 🤞 |
Anyone try this on 8.3.2 yet? |
@michaellady you're welcome to clone the project here https://github.com/iwllyu/UITestCoverageBug and try it out on 8.3.2 |
Hi @iwllyu, I'm still seeing the command line issue on 8.3.2 |
thanks, I've updated the readme in that project |
I looked into fixing this. Found the difference in logs that @thelvis4 did. Figured I'd see if there was an undocumented flag I could pass to xcodebuild to use the LLDB launcher instead of PosixSpawn. Decompiling didn't find anything, so I decided to force it myself to see if it would work. Modifying xcode build to use LLDB instead of PosixSpawn indeed works, but its fragility won't be an ideal solution to implement into most projects. Most of us are probably stuck waiting for Apple to implement a fix, so file a Radar! Proof of Concept commands:
Proof of concept code for injection as a library: https://gist.github.com/JohnCoates/605319ab31c0aa05c77973481aeae4b2 |
https://github.com/coryalder is reporting that this is fixed in XCode9 |
Woot!
…On Tue, Jun 13, 2017 at 12:09 PM iwllyu ***@***.***> wrote:
https://github.com/coryalder is reporting that this is fixed in XCode9
iwllyu/UITestCoverageBug#4
<iwllyu/UITestCoverageBug#4>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#22 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA5JKml8y7RqQN81LL9qN3_YHjEWV16Xks5sDt6AgaJpZM4IKWuQ>
.
|
Amazing work by the dev tools team in this release 👏 |
OMG |
Oops we may have jumped the gun. iwllyu/UITestCoverageBug#4 (comment) tl;dr: it looks like both xcode and xcodebuild are not covering UI tests. xcode could have erroneously reporting coverage before and Apple intends for UI tests not to be part of code coverage. or it's a bug Well, at least they're consistent now |
@iwllyu Any update in this?? I'm running Xcode 9 and after ran all the tests the UI Tests are missed from the xcoverage report. Xcov is only reporting the unit tests. |
Xcode Version 9.3 beta (9Q98q) seems resolves issue with missed UI Tests coverage as long as |
I've verified this is indeed working again 🎉 |
Sorry to comment on this after so long, but this is the only issue that relates to the problem I'm having. Not even unit tests are running right now. If I run them in Xcode, it generates coverage and does everything as expected. If it's run via run_tests / scan, it doesn't appear that unit tests are even running, even though it has output like this:
Can anybody help with this? |
This sounds like you should open a new issue if this was closed that long
ago. I don’t even do mobile devops anymore, lol.
…On Fri, Jan 26, 2024 at 1:20 AM stephen-oconnor-cp ***@***.***> wrote:
Sorry to comment on this after so long, but this is the only issue that
relates to the problem I'm having.
Not even unit tests are running right now. If I run them in Xcode, it
generates coverage and does everything as expected. If it's run via
run_tests / scan, it doesn't appear that unit tests are even running, even
though it has output like this:
[10:13:55]: Running Tests: ▸ Touching CameraDummyTests.xctest (in target 'CameraDummyTests' from project 'CameraDummy')
[10:13:55]: ▸ Processing empty-BuildPipeSandbox.plist
[10:13:55]: ▸ Processing empty-BuildPipeSandboxTests.plist
[10:13:56]: ▸ Linking BuildPipeSandboxTests
[10:13:56]: Running Tests: ▸ Touching BuildPipeSandboxTests.xctest (in target 'BuildPipeSandboxTests' from project 'BuildPipeSandbox')
[10:13:56]: Running Tests: ▸ Touching BuildPipeSandbox.app (in target 'BuildPipeSandbox' from project 'BuildPipeSandbox')
[10:13:56]: ▸ Linking BuildPipeSandboxUITests
[10:13:56]: ▸ Processing empty-BuildPipeSandboxUITests.plist
[10:13:56]: ▸ Copying Info.plist
[10:13:56]: Running Tests: ▸ Touching BuildPipeSandboxUITests.xctest (in target 'BuildPipeSandboxUITests' from project 'BuildPipeSandbox')
[10:13:56]: ▸ Test build Succeeded
[10:13:56]: Copying .xctestrun file
Can anybody help with this?
—
Reply to this email directly, view it on GitHub
<#22 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHESKX33DYIMN437XYJUI3YQNYPFAVCNFSM4CBJNOIKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJRGE3TENRZHE4Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Everyone I run xcov I am getting 0%
I created a new Project with some quick code and Xcode is showing the count beside each function inside the IDE.
My Command
`xcov -s AutoTest -p ./AutoTest.xcodeproj -o ./build/reports/xcov_output
I get the following Output
I've attached a screenshot, if you need any more info let me know.
I love the html output, just hope the Cover Coverage gets corrected
The text was updated successfully, but these errors were encountered: