Code coverage

[markelog]

Evaluate and prototype implementation of code coverage tool. Maybe not even on Core at first, Sizzle?

/cc @timmywil, @mzgol, @gibson042, @jaubourg, @dmethvin

[markelog]

and /cc @gnarf

[scottgonzalez]

We already have multiple projects doing this.

@arschmitz @jzaefferer

[arschmitz]

We use coveralls it was really really noisy at first so we disabled the pr comments ( looks like qunit has as well ) which means we lose a lot of the value. I was just looking at they have notification settings with coverage ranges for them now so this may not be as much of an issue any more.

Previously every new commit on a PR triggered a coverage comment, which was a lot of noise when there was often absolutely no actual change in coverage.

[markelog]

In mobile? Istanbul? In what envs do you run it? And How :-)

[arschmitz]

@markelog Yes on mobile. We use grunt-coveralls https://www.npmjs.com/package/grunt-coveralls both locally and via travis.

[markelog]

Yeah, grunt-coveralls is a plugin for grunt, not a code coverage tool, and coveralls is service for representation code coverage data, which is also not code coverage tool :-)

Just check it out - you use istanbul on phantom.

[gnarf]

getting coverage on jQuery Core would be so.... tricky

so many "forks" that require a specific actual browser to hit...

[gibson042]

But those problems are apparently solved; istanbul-middleware enables a mode of operation that aggregates results: https://github.com/gotwarlost/istanbul-middleware#client-side-coverage .

So it should be as simple as spinning up a node.js server with istanbul-middleware and making sure that the test swarm gets jquery.js from it, then downloading/looking at the coverage results before tearing down the server when its run is complete.

[listepo]

+1

[stevepeak]

Full disclosure I'm the founder of Codecov

We would love to see jQuery use Codecov as its coverage provider. I would love to share about the many reasons why I believe Codecov are great fit and industry leading in code coverage. Anything I can do to help this transition I would make myself very available. Thank you!

[gnarf]

Anything I can do to help this transition I would make myself very available.

Help implement it in a PR and I'm sure we can work together on it! A couple of questions:

• What does it take for us to use it? (we don't use travis)
• Can we use it in the jenkins / testswarm / qunit setup that we have?
• Do you have some example reports to look at, could you run it against jQuery yourself and link us to a report?

I'm sure others will have some more questions :)

[stevepeak]

What does it take for us to use it? (we don't use travis)

Any CI method works. We have two primary uploading methods for javascript: the native method and the global method. I'll look into creating a pull request, I'll mention you when I create it. Thank you @gnarf

Can we use it in the jenkins / testswarm / qunit setup that we have?

Jenkins: yes. Qunit: yes. Testswarm: should (I have not used this before). We do merge coverage reports automatically and keep record of each one. See here for example (Click on the Unified dropdown).

Do you have some example reports to look at, could you run it against jQuery yourself and link us to a report?

I can reference our customer repos, but I have not yet ran coverage against jQuery yet. I will try to attempt this during my pull request.

I'm sure others will have some more questions :)

I'm very happy to help out!

[gnarf]

I'm very happy to help out!

Feel free to ping me (gnarf) in freenode IRC #jquery-dev with any questions/help you'd need from me here.

[timmywil]

Closing in favor of tracking in the Roadmap.