Add support to profdata file format #92

viteinfinite · 2015-06-15T17:43:50Z

Hey,
I'm adding experimental support to the profdata file format (i.e. the format used in Xcode 7 beta).

The current implementation calls llvm-cov and reads the subsequent output.
Two new configuration parameters have been added:

scheme: allows the selection of a scheme; it is used to find the folder containing the correct instrumentation data
input-format: in case the value is profdata, Slather will look for the Coverage.profdata file inside the build directory

Branch information is not included at the moment as I still have to figure out how (and if) this information is actually included in the Coverage.profdata file.

Please feel free to discuss or inquiry about any implementation detail.

sync · 2015-06-18T01:46:47Z

also see Fix broken path viteinfinite/slather#1 (minor fix)
usage: slather coverage --input-format profdata --scheme YauShop YauShop.xcodeproj
output extract with Xcode 7 coverage:

YauShop/Utils/TegQ.swift: 4 of 10 lines (40.00%)
YauShop/Utils/TegRecycler.swift: 0 of 11 lines (0.00%)
YauShop/Utils/TegRegex.swift: 12 of 15 lines (80.00%)
YauShop/Utils/TegScreenSize.swift: 0 of 22 lines (0.00%)
YauShop/Utils/TegScreenType.swift: 0 of 3 lines (0.00%)
YauShop/Utils/TegSecureRandom.swift: 11 of 15 lines (73.33%)
YauShop/Utils/TegString.swift: 27 of 27 lines (100.00%)
YauShop/Utils/TegTextFileLoader.swift: 4 of 4 lines (100.00%)
YauShop/Utils/TegTickTock.swift: 0 of 23 lines (0.00%)
YauShop/Utils/TegUIColor.swift: 37 of 37 lines (100.00%)
YauShop/Utils/TegUrlEncoder.swift: 5 of 5 lines (100.00%)
YauShop/Utils/TegViewPosition.swift: 0 of 25 lines (0.00%)
YauShop/Utils/UI/KeyboardObserver.swift: 31 of 34 lines (91.18%)
YauShop/Utils/UI/RoundedButton.swift: 7 of 10 lines (70.00%)
YauShop/Utils/UI/RoundedView.swift: 10 of 10 lines (100.00%)
Test Coverage: 39.79%
Slathered

Fix broken path

viteinfinite · 2015-06-18T18:05:34Z

@sync thank you for the usage sample!

sync · 2015-06-18T23:58:13Z

right now it seems that:

the JSON exporter doesn't work
coverall doesn't work
cobertura is working

viteinfinite · 2015-06-21T08:50:51Z

Thank you for the feedback, I've been working on some improvements. I'll push the modifications in some hours.

…te/slather into feature-temp-profdata * 'feature-temp-profdata' of https://github.com/viteinfinite/slather: Remove useless file Fix tests Remove useless files Fix unit tests Add more precise path handling Add scheme and ignore Export coverage base info into modules Fix coveralls_spec Minor fixes in spec_helper Re-add coveralls support Re-add coveralls support Add tests [WIP] Refactor interface of coverage_file [WIP] Add unit tests [WIP] Add experimental support to swift

Add JSON support to profdata format

marklarr · 2015-06-22T07:05:09Z

This is cool -- I'll check it out this week!

Fix whitespace issues

viteinfinite · 2015-08-05T16:37:40Z

@marklarr Any update on this?

* petester42-master: Add spec for framework coverage Fix llvm-cov path and add framework coverage

mjdetullio · 2015-08-17T19:09:17Z

Built and installed the Gem from source. Using the Cobertura reporting now and recording it in Jenkins. Works great.

/usr/local/bin/slather \
        coverage \
        --cobertura-xml \
        --output-directory "${WORKSPACE}/build/coverage-reports" \
        --input-format profdata \
        --build-directory "${WORKSPACE}/DerivedData" \
        --source-directory . \
        --ignore '../**' \
        --ignore 'SomeSource/generated/**' \
        --ignore 'Tests/**' \
        --scheme "MyTests" \
        MyProject.xcodeproj

One thing I noticed is it has to be run from the source root, otherwise the rest of the tree is displayed on the report, regardless of the --source-directory option. Just ended up doing a pushd/popd into the source root and setting the source dir as the current dir.

marklarr · 2015-08-18T17:03:57Z

Hey @viteinfinite -- I was out on vacation for a couple weeks. I'll get to looking at this later this week!

marklarr · 2015-08-21T17:02:11Z

@viteinfinite Is this ready to be merged? It looks good to me. We just need to resolve the conflicts.

viteinfinite · 2015-08-21T22:49:39Z

@marklarr I can close this PR and submit a new one resolving the conflicts or you manually merge them. Either way, the work on the feature itself is completed.

…te/slather into feature-temp-profdata * 'feature-temp-profdata' of https://github.com/viteinfinite/slather: Remove useless file Fix tests Remove useless files Fix unit tests Add more precise path handling Add scheme and ignore Export coverage base info into modules Fix coveralls_spec Minor fixes in spec_helper Re-add coveralls support Re-add coveralls support Add tests [WIP] Refactor interface of coverage_file [WIP] Add unit tests [WIP] Add experimental support to swift

viteinfinite · 2016-01-07T09:35:06Z

@tarbrain thanks for your feedback. I totally agree with you.
I'll start working on this next week.

…pecs Update specs to test against profdata output instead of gcov

viteinfinite · 2016-01-18T17:31:31Z

@tarbrain So here it is. These last commits modify the specs in order to test against the profdata output.

Also, a verbose mode (--verbose or -v) has been added which avoids printing the debug lines by default.

Finally, platform headers, such as Library/Frameworks/XCTest.framework/Headers/XCTestAssertionsImpl.h, are now filtered.

jkrumow · 2016-01-18T20:04:59Z

@viteinfinite Looks sweet. I would say we all give it a try and if nobody finds a bug I will merge it soon.

Good work 🎉

Add support to profdata file format

jkrumow · 2016-01-19T19:59:22Z

@viteinfinite Its done!
@marklarr @ayanonagon You can release a new gem version 👍

viteinfinite · 2016-01-20T01:11:00Z

🎉🎉🎉

viteinfinite · 2016-01-26T14:48:20Z

@tarbrain BTW, gem's not pushed yet?

jkrumow · 2016-01-26T16:26:49Z

I can not push the gem since I do not belong to the developers at venmo.

We should ask @marklarr or @ayanonagon again.

ayanonagon · 2016-01-26T16:43:03Z

@tarbrain Is your email the one listed on GitHub? If so I’ll add you as an owner to the gem. Thanks for helping us out. We really appreciate it! 😃

jkrumow · 2016-01-26T18:58:17Z

@ayanonagon @marklarr Thanks for adding me to the list.

But please do not rely on me for maintaining this repository. I wont have the time. I am just one of the many professional slather users who are waiting for the important profdata feature and feel somewhat let down by the venmo developers. You have a big user base and you should take responsibility for that.

Regards
tarbrain

richardbuckle · 2016-01-27T08:51:40Z

As a user, I do hope that the gem will be pushed soon. It has been quite a long wait.

I've been holding off on publishing a blog post about getting Xcode 7 to play nice with Jenkins because I want my instructions for installing slather to be simply gem install slather. It doesn't feel right to send people to a fork of the repo but maybe that would be better than waiting much longer.

ayanonagon · 2016-02-03T21:44:24Z

@tarbrain We’re very sorry for the wait. It’s clear that we haven’t done a great job in maintaining this project. The main issues are that nobody on the current iOS team is fluent in Ruby and also that we are not currently using this project in our own workflow. It’s unfair to make other developers wait because of us, so we apologize again for that.

On a happier note, our friend @neonichu has graciously offered to pick up maintaining the project. As you might have noticed, we pulled it out into a separate GitHub organization as well. Anyway, slather is in more capable hands now, so we’re excited to see where the community takes it. 🎉

Thanks!

AliSoftware · 2016-02-03T22:18:03Z

@ayanonagon never apologize for doing OpenSource and giving such a great tool to the community for free 😉

@neonichu thx for picking up on the project maintenance 🎉

neonichu · 2016-02-03T22:23:18Z

@ayanonagon @marklarr 🍻

neonichu · 2016-02-03T23:40:10Z

Allright, created a milestone for 2.0 https://github.com/SlatherOrg/slather/milestones/2.0.0, once the things in there are addressed, we can

viteinfinite · 2016-02-04T00:02:21Z

@ayanonagon thanks for the update.
@neonichu, may your crafty hands save us from bugs dues to uncovered code.

jkrumow · 2016-02-04T11:17:02Z

@ayanonagon @neonichu
Thank you for the response! I think making slather an organization is a good decision and makes it easier to participate. Very good!

Add experimental support to profdata file format

b1de838

viteinfinite mentioned this pull request Jun 15, 2015

Support Xcode 7's code coverage mapping format #89

Closed

fix broken path

b305ed8

Merge pull request #1 from sync/feature-profdata

d3bece9

Fix broken path

viteinfinite added 3 commits June 21, 2015 12:10

Merge pull request #2 from viteinfinite/feature-temp-profdata

8558508

Add JSON support to profdata format

Add JSON support to profdata file format

32dab65

viteinfinite added 2 commits July 10, 2015 20:54

Fix whitespace issues

c70948f

Merge pull request #3 from viteinfinite/fix-whitespaces

7236739

Fix whitespace issues

pmairoldi and others added 3 commits August 8, 2015 14:39

fix llvm-cov path and add framework coverage

9e5724b

Add spec for framework coverage

c5d2b70

Merge branch 'petester42-master' into feature-profdata

61f0098

* petester42-master: Add spec for framework coverage Fix llvm-cov path and add framework coverage

viteinfinite mentioned this pull request Aug 10, 2015

fix llvm-cov path and add framework coverage viteinfinite/slather#4

Closed

viteinfinite and others added 6 commits August 29, 2015 16:32

Add experimental support to profdata file format

5cd2399

fix broken path

9aa2af2

Fix whitespace issues

95a067b

fix llvm-cov path and add framework coverage

e64a7b7

Add spec for framework coverage

8aa1c12

cuteiosdev mentioned this pull request Aug 29, 2015

Feature profdata #99

Closed

viteinfinite added 6 commits January 18, 2016 18:08

[command] Add verbose mode

3442a20

[profdata] Ignore platform files

2930bef

[command] Add spec for verbose mode

ab7004c

[profdata] Add specs for ignoring platform files

c460c4f

[specs] Update specs to test against profdata output instead of gcov

edc92c4

Merge pull request #17 from viteinfinite/feature-profdata-flavoured-s…

6a19585

…pecs Update specs to test against profdata output instead of gcov

viteinfinite mentioned this pull request Jan 18, 2016

Profdata Framework Fix viteinfinite/slather#11

Open

jkrumow pushed a commit that referenced this pull request Jan 19, 2016

Merge pull request #92 from viteinfinite/feature-profdata

7a0921b

Add support to profdata file format

jkrumow merged commit 7a0921b into SlatherOrg:master Jan 19, 2016

cutz mentioned this pull request Jan 20, 2016

html report coloring missed lines as covered #122

Closed

neonichu mentioned this pull request Feb 3, 2016

Swift test coverage. #39

Closed

larslockefeer mentioned this pull request Feb 4, 2016

Add binary_basename configuration option #128

Merged

yas375 mentioned this pull request May 1, 2016

�� in results #199

Closed

drewcrawford mentioned this pull request Jul 14, 2016

Code coverage AnarchyTools/atbuild#30

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support to profdata file format #92

Add support to profdata file format #92

viteinfinite commented Jun 15, 2015

sync commented Jun 18, 2015

viteinfinite commented Jun 18, 2015

sync commented Jun 18, 2015

viteinfinite commented Jun 21, 2015

marklarr commented Jun 22, 2015

viteinfinite commented Aug 5, 2015

mjdetullio commented Aug 17, 2015

marklarr commented Aug 18, 2015

marklarr commented Aug 21, 2015

viteinfinite commented Aug 21, 2015

viteinfinite commented Jan 7, 2016

viteinfinite commented Jan 18, 2016

jkrumow commented Jan 18, 2016

jkrumow commented Jan 19, 2016

viteinfinite commented Jan 20, 2016

viteinfinite commented Jan 26, 2016

jkrumow commented Jan 26, 2016

ayanonagon commented Jan 26, 2016

jkrumow commented Jan 26, 2016

richardbuckle commented Jan 27, 2016

ayanonagon commented Feb 3, 2016

AliSoftware commented Feb 3, 2016

neonichu commented Feb 3, 2016

neonichu commented Feb 3, 2016

viteinfinite commented Feb 4, 2016

jkrumow commented Feb 4, 2016

Add support to profdata file format #92

Add support to profdata file format #92

Conversation

viteinfinite commented Jun 15, 2015

sync commented Jun 18, 2015

viteinfinite commented Jun 18, 2015

sync commented Jun 18, 2015

viteinfinite commented Jun 21, 2015

marklarr commented Jun 22, 2015

viteinfinite commented Aug 5, 2015

mjdetullio commented Aug 17, 2015

marklarr commented Aug 18, 2015

marklarr commented Aug 21, 2015

viteinfinite commented Aug 21, 2015

viteinfinite commented Jan 7, 2016

viteinfinite commented Jan 18, 2016

jkrumow commented Jan 18, 2016

jkrumow commented Jan 19, 2016

viteinfinite commented Jan 20, 2016

viteinfinite commented Jan 26, 2016

jkrumow commented Jan 26, 2016

ayanonagon commented Jan 26, 2016

jkrumow commented Jan 26, 2016

richardbuckle commented Jan 27, 2016

ayanonagon commented Feb 3, 2016

AliSoftware commented Feb 3, 2016

neonichu commented Feb 3, 2016

neonichu commented Feb 3, 2016

viteinfinite commented Feb 4, 2016

jkrumow commented Feb 4, 2016