Provide coverage report format Jenkins can consume

[Iristyle]

I think this is the right spot for the request.

Istanbul was recently integrated into Testacular, and it's pretty fantastic. It would be nice to see a format that Jenkins can natively handle, rather than lcov and/or HTML.

It looks like people tend to favor using the Cobertura plugin... and if they have incoming LCov, they convert.
I found an lcov-tocobertura tool, but it's in Python :( Not sure if there are other / better options here I just haven't come across.

Also see post on SO regarding lcov in Hudson / Jenkins.

/cc @taichi

I opened an issue at Testacular to track this as well.
https://github.com/vojtajina/testacular/issues/164#issuecomment-10298818

Thanks for the consideration!

[gotwarlost]

Yes, this is the right spot. Let me take a look.

For now would you be able to use a post-processing step using the lcov-cobertura tool to get by for a few days?

[gotwarlost]

@Iristyle I added the cobertura output but don't have a good way to test it. I've made sure that the output is well-formed XML and matches the output of the file generated by the lcov to cobertura python tool.

Try it out and see if it works for you.

$ istanbul report cobertura

should create a cobertura-coverage.xml file.

Available in version v0.1.21

[Iristyle]

Wow, that was fast! Thanks a bunch.

I will try to get to checking this out later tonight.

[bahmutov]

Nice! I was looking to convert lcov to covertura when I saw the commit. Unfortunately Jenkins crashed on simple report. Here is the cobertura report xml I am trying to process using Cobertura plugin.

<?xml version="1.0" ?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage lines-valid="14"  lines-covered="11"  line-rate="0.7857"  branches-valid="2"  branches-covered="1"  branch-rate="0.5"  timestamp="1352925473714" complexity="0" version="0.1">
<sources />
<packages>
    <package name="test1"  line-rate="0.7857"  branch-rate="0.5" >
    <classes>
        <class name="code.js"  filename="C:\Program Files (x86)\Jenkins\jobs\istanbulTest1\workspace\js\coverage\test1\code.js"  line-rate="0.7857"  branch-rate="0.5" >
        <methods>
            <method name="foo"  hits="1"  />
            <method name="bar"  hits="0"  />
        </methods>
        <lines>
            <line number="1"  hits="1"  branch="false" />
            <line number="2"  hits="1"  branch="false" />
            <line number="3"  hits="1"  branch="false" />
            <line number="6"  hits="1"  branch="false" />
            <line number="7"  hits="0"  branch="false" />
            <line number="8"  hits="0"  branch="false" />
            <line number="11"  hits="1"  branch="false" />
            <line number="12"  hits="1"  branch="false" />
            <line number="14"  hits="1"  branch="false" />
            <line number="16"  hits="1"  branch="true"  condition-coverage="50% (1/2)" />
            <line number="17"  hits="1"  branch="false" />
            <line number="19"  hits="0"  branch="false" />
            <line number="22"  hits="1"  branch="false" />
            <line number="23"  hits="10"  branch="false" />
        </lines>
        </class>
    </classes>
    </package>
</packages>
</coverage>

and here is the Jenkins exception:
Publishing Cobertura coverage report...
15:37:53 Publishing Cobertura coverage results...
15:37:53 ERROR: Publisher hudson.plugins.cobertura.CoberturaPublisher aborted due to exception
15:37:53 java.lang.NullPointerException
15:37:53 at java.util.regex.Matcher.getTextLength(Unknown Source)
15:37:53 at java.util.regex.Matcher.reset(Unknown Source)
15:37:53 at java.util.regex.Matcher.(Unknown Source)
15:37:53 at java.util.regex.Pattern.matcher(Unknown Source)
15:37:53 at hudson.plugins.cobertura.CoberturaXmlHandler.buildMethodName(CoberturaCoverageParser.java:241)

Hope it can help stabilize istanbul.

[gotwarlost]

@bahmutov could you run an equivalent test of taking the lcov output, converting it into cobertura format using the python tool?

If that works, you could attach the XML generated by that tool and I can see what need to be fixed. Thanks.

[gotwarlost]

Looks like the signature required attribute (from the latest DTD) is missing causing the NPE. I doubt the python tool will do any better. However, since it produces XML conforming to an older DTD, it may work if the coverage parser has conditional processing based on DTD version.

[bahmutov]

Hmm, I guess coberture plugin is broken for this.

On Nov 14, 2012, at 3:55 PM, Krishnan Anantheswaran notifications@github.com wrote:

Looks like the signature required attribute (from the latest DTD) is missing causing the NPE. I doubt the python tool will do any better. However, since it produces XML conforming to an older DTD, it may work if the coverage parser has conditional processing based on DTD version.

—
Reply to this email directly or view it on GitHub.

[gotwarlost]

@bahmutov Yep, that's it - the missing signature attribute. Fixed the report to add a dummy signature attribute to every method tag, set up a dummy Jenkins project to consume it and it works (fails without the fix). I'm pretty confident version 0.1.22 will work for you. Let me know either way.

[gotwarlost]

Also noticed that the method coverage doesn't work as expected for the cobertura report, since it needs detailed info about the lines and branches within the method and istanbul does not track anything at this level. Line and branch information should be consistent with other reporting formats though.

[bahmutov]

Excellent thank, I am impressed with istanbul, it is a great tool

On Nov 14, 2012, at 4:22 PM, Krishnan Anantheswaran notifications@github.com wrote:

@bahmutov Yep, that's it - the missing signature attribute. Fixed the report to add a dummy signature attribute to every method tag, set up a dummy Jenkins project to consume it and it works (fails without the fix). I'm pretty confident version 0.1.22 will work for you. Let me know either way.

—
Reply to this email directly or view it on GitHub.

[gotwarlost]

@bahmutov thanks!