Configuring istanbul with mocha #44

Closed
dankohn opened this Issue Apr 9, 2013 · 25 comments

Projects

None yet
@dankohn
dankohn commented Apr 9, 2013

Istanbul works flawlessly with jasmine-node, using the command: `istanbul cover jasmine-node test'

Is it possible to use istanbul with mocha, in a similar way. I get the following:

$ istanbul cover mocha -u exports -R spec

fs.js:684
  return binding.stat(pathModule._makeLong(path));
                 ^
Error: ENOENT, no such file or directory 'exports.js'
    at Object.fs.statSync (fs.js:684:18)
    at lookupFiles (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:390:17)
    at spinner (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:268:24)
    at Array.forEach (native)
    at Object.<anonymous> (/usr/local/share/npm/lib/node_modules/mocha/bin/_mocha:267:6)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3
No coverage information was collected, exit without writing coverage information

I tried using https://github.com/arikon/mocha-istanbul as the reporter but got a similar error.

@gotwarlost
Owner

So there are two issues with the command you are running:

  • The -u exports and -R spec are considered to be istanbul flags rather than mocha flags and are silently ignored.

To fix this, use:

istanbul cover mocha -- -u exports -R spec # note the double-dash between mocha and options
  • mocha forks for running tests and istanbul will not hook require in the sub-process

At this point, mocha will at least honor the options but it still won't provide coverage. That's because the mocha executable forks the _mocha executable to run tests (to be able to support node options etc.)

So what you need is:

istanbul cover _mocha -- -u exports -R spec # note the leading underscore 

At this point everything should work. Let me know how it goes.

@dankohn
dankohn commented Apr 9, 2013

Yes, this worked perfectly. Thanks so much. istanbul is amazing.

@dankohn dankohn closed this Apr 9, 2013
@paulbjensen

Thank you for adding this, been trying to find a good guide on how to achieve this combo

@meritt
meritt commented Jan 3, 2014

I have another issue similar with this. How to send --harmony flag to _mocha. I tried many options, but nothing to work. With mocha --harmony flag work perfectly, but istanbul stops working.

@gotwarlost
Owner

You can't directly do this with the istanbul command (because that runs node implicitly).

Assuming the original command was:

istanbul cover _mocha -- -u exports -R spec 

turn it into:

node --harmony node_modules/istanbul/lib/cli.js cover _mocha -- -u exports -R spec
@meritt
meritt commented Jan 4, 2014

Thanks, it helped partially, but now I get SyntaxError: Unexpected strict mode reserved word.

/Users/simonenko/Sites/projects/yamb/lib/yamb/proto/methods/save.js:9
XiYtwg.b['10'][0]++;__cov_H$BUi8TOdhvMxeE7XiYtwg.s['37']++;let data=yield asyn
                                                                    ^^^^^

I tried to run istanbul with node --harmony --use_strict, but now I've got istanbul error.

/Users/simonenko/Sites/projects/yamb/node_modules/istanbul/node_modules/mkdirp/index.js:9
        mode = 0777 & (~process.umask());
               ^^^^
Octal literals are not allowed in strict mode.
SyntaxError: Octal literals are not allowed in strict mode.
@gotwarlost
Owner

Did you try adding a "use strict";' at the top of the file where you are usinglet`? (instead of globally turning on strict I mean)

@meritt
meritt commented Jan 5, 2014

Yeah, but the feeling that it disappears. All my files have "use strict" at the top of the file.

@gotwarlost
Owner

You are most likely using a version of istanbul that does not handle file level strict mode correctly. Please use the harmony branch of this source tree to get the correct version. This also depends on the harmony branch of the esprima source tree so you will get a consistent set of dependencies. Not all es6 constructs are supported by the tools at this point so you may still have issues. Let me know what you find.

@meritt
meritt commented Jan 6, 2014

All this time I used the harmony branch. And all my files have "use strict" at the top of the file. If I run mocha without istanbul all tests passed. See my dependency versions:

  , "devDependencies": {
        "mocha": "~1.16"
      , "should": "~2.1"
      , "coveralls": "~2.6"
      , "istanbul": "git://github.com/gotwarlost/istanbul.git#harmony"
    }
@Qard
Qard commented Jan 19, 2014

I can't seem to get --harmony working either. I tried node --harmony node_modules/.bin/_mocha and it worked fine, but using node --harmony node_modules/.bin/istanbul cover _mocha on the harmony branch just gives me this long error. Not entirely sure how that qualifies as an unexpected identifier...

/apps/test-app/app/controllers/users.js:9
sole.log('before user');__cov_82tiDfzAM405_Tb8tPefzw.s['3']++;yield next;};__c
                                                                    ^^^^
No coverage information was collected, exit without writing coverage information
SyntaxError: Unexpected identifier
    at exports.runInThisContext (vm.js:69:16)
    at Module._compile (module.js:432:25)
    at Object.Module._extensions..js (/apps/test-app/node_modules/istanbul/lib/hook.js:99:20)
    at Module.load (module.js:349:32)
    at Function.Module._load (module.js:305:12)
    at Module.require (module.js:357:17)
    at require (module.js:373:17)
    at Object.<anonymous> (/apps/test-app/test/controllers.users.js:1:82)
    at Module._compile (module.js:449:26)
    at Module._extensions..js (module.js:467:10)
    at Object.Module._extensions..js (/apps/test-app/node_modules/istanbul/lib/hook.js:101:13)
    at Module.load (module.js:349:32)
    at Function.Module._load (module.js:305:12)
    at Module.require (module.js:357:17)
    at require (module.js:373:17)
    at /apps/test-app/node_modules/mocha/lib/mocha.js:172:27
    at Array.forEach (native)
    at Mocha.loadFiles (/apps/test-app/node_modules/mocha/lib/mocha.js:169:14)
    at Mocha.run (/apps/test-app/node_modules/mocha/lib/mocha.js:356:31)
    at Object.<anonymous> (/apps/test-app/node_modules/mocha/bin/_mocha:359:16)
    at Module._compile (module.js:449:26)
    at Module._extensions..js (module.js:467:10)
    at Object.Module._extensions..js (/apps/test-app/node_modules/istanbul/lib/hook.js:101:13)
    at Module.load (module.js:349:32)
    at Function.Module._load (module.js:305:12)
    at Function.Module.runMain (module.js:490:10)
    at runFn (/apps/test-app/node_modules/istanbul/lib/command/common/run-with-cover.js:110:16)
    at /apps/test-app/node_modules/istanbul/lib/command/common/run-with-cover.js:213:17
    at /apps/test-app/node_modules/istanbul/lib/util/file-matcher.js:52:16
    at /apps/test-app/node_modules/istanbul/lib/util/file-matcher.js:35:9
    at Object.next (/apps/test-app/node_modules/istanbul/node_modules/fileset/lib/fileset.js:41:14)
    at Fileset.EventEmitter.emit (events.js:101:17)
    at Fileset.Glob._processEmitQueue (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:313:10)
    at Fileset.Glob._pushMatch (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:287:8)
    at Fileset.Glob.emitMatch (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:292:10)
    at Fileset.Glob._finish (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:220:8)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:178:27)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:447:32)
    at Fileset.cb (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:334:11)
    at Fileset.<anonymous> (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:414:14)
    at Fileset.Glob._afterReaddir (/apps/test-app/node_modules/istanbul/node_modules/fileset/node_modules/glob/glob.js:657:17)
    at Object.oncomplete (fs.js:97:15)
@gotwarlost
Owner

@Qard - that actually looks like the instrumenter emitted some bad code. Is there a way you can share the users.js file with me? Or if you could provide a reproducible test case that would be great too.

@fengmk2
fengmk2 commented Mar 3, 2014

Awesome!

@fengmk2 fengmk2 added a commit to koajs/koa-gzip that referenced this issue Mar 3, 2014
@fengmk2 fengmk2 use istanbul replace blanket. gotwarlost/istanbul#44 9af6950
@geertweening geertweening added a commit to ripple/ripple-rest that referenced this issue Oct 1, 2014
@geertweening geertweening [TASK] fix coverage by running `_mocha`
as suggested here gotwarlost/istanbul#44
4866370
@anu-srivastava

Hi, I am trying to use Istanbul but I am getting following error:-
istanbul cover _mocha -- -u exports -R spec

C:\Users\asrivastava1\AppData\Roaming\npm_mocha.CMD:1
(function (exports, require, module, __filename, __dirname) { @IF EXIST "%~dp0
^
No coverage information was collected, exit without writing coverage information

SyntaxError: Unexpected token ILLEGAL
at Module._compile (module.js:439:25)
at Module._extensions..js (module.js:474:10)
at Object.Module._extensions..js (C:\Users\asrivastava1\AppData\Roaming\npm
node_modules\istanbul\lib\hook.js:102:13)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at runFn (C:\Users\asrivastava1\AppData\Roaming\npm\node_modules\istanbul\li
b\command\common\run-with-cover.js:114:16)
at C:\Users\asrivastava1\AppData\Roaming\npm\node_modules\istanbul\lib\comma
nd\common\run-with-cover.js:232:17
at C:\Users\asrivastava1\AppData\Roaming\npm\node_modules\istanbul\lib\util
file-matcher.js:56:16
at C:\Users\asrivastava1\AppData\Roaming\npm\node_modules\istanbul\lib\util
file-matcher.js:35:9

Can you please guide me what am I doing wrong

@gotwarlost
Owner

Using _mocha directly does not work on windows. Please use the full path to the JS file instead

@anu-srivastava

Yeah I just figured that if I use following command then it does run :-
"./node_modules/.bin/istanbul" cover "./node_modules/mocha/bin/mocha" -- server/*/_.js -R spec

@TheSavior TheSavior referenced this issue in mochajs/mocha Apr 8, 2015
Closed

2.2.3 no longer exposes suite global #1645

@ublue1024

Thank you, #44 also solved my problem. In Windows 7, I just use
istanbul cover {my_npm_path}\node_modules\mocha\bin_mocha

@otuome
otuome commented Apr 20, 2015

hey @gotwarlost, i'm on osx 10 and running the following script for a simple express app via npm test results in "No coverage information was collected, exit without writing coverage information" even though the tests run successfully:

istanbul cover _mocha -- -R tap \"test/*-test.js\"

what's the best way to debug the reason that coverage.json is not being generated?

curious b/c the following does generate a coverage.json, as well as, the CLI summary output:

istanbul cover app.js

@rstacruz rstacruz referenced this issue in arikon/mocha-istanbul Jul 6, 2015
Open

PSA: you can use mocha with istanbul directly #7

@matthewloring matthewloring referenced this issue in matthewloring/cloud-trace-nodejs Aug 27, 2015
@matthewloring matthewloring Coveralls runs with travis 6abfa84
@gajus gajus referenced this issue Sep 6, 2015
Closed

What is _mocha? #436

@KurtPattyn KurtPattyn added a commit to KurtPattyn/karl that referenced this issue Oct 1, 2015
@KurtPattyn KurtPattyn Change istanbul call with call to cli.js
See: gotwarlost/istanbul#44
Now karl can be used with node 0.12 and later again
77992c7
@KurtPattyn KurtPattyn added a commit to KurtPattyn/kimbu that referenced this issue Oct 1, 2015
@KurtPattyn KurtPattyn Change istanbul call with call to cli.js
See: gotwarlost/istanbul#44
Now kimbu can be used with node 0.12 and later
9017040
@KurtPattyn KurtPattyn added a commit to KurtPattyn/krew that referenced this issue Oct 1, 2015
@KurtPattyn KurtPattyn Change istanbul call with call to cli.js
See: gotwarlost/istanbul#44
Now krew can be used with node 0.12 and later
dc0cae9
@Om-sudhir

Hi Team,
Even we tried with full path in windows 7 m/c ,it doesn't works. All Tests were executed and passed ,No coverage information was collected, exit without writing coverage information.

istanbul cover C:\Users\user_sam\AppData\Roaming\npm\node_modules\mocha\bin\mocha -- -R spec

D:\BaseRepo3\FirstNodejsWebApp>istanbul cover D:\KBaseRepo3\FirstNodejsWebApp\node_modules\mocha\bin_mocha -- -R spec

KB unit test
V should return all article (91ms)
V should return home Page (173ms)
V should return article by TagName
V should return article by ID
UnitTestTag added to the article with id = 564d9eddd33338af2dd709d
V should add newtag by iD AND TAG
V should list all the available TAGs
V should list of top 5 articles
V should list articles matching all tags

8 passing (785ms)

No coverage information was collected, exit without writing coverage information

Please help!!

@xjamundx

@Om-sudhir try linking to the _mocha file instead of mocha

@ORESoftware

Just saying - it would be a lot cooler if mocha tests could be run with the plain old Node executable.

@xjamundx
xjamundx commented Mar 9, 2016

@ORESoftware you can use https://github.com/bcoe/nyc for that and it works really well!

@ndaidong

@xjamundx thanks.

My problem with istanbul is it does not return any coverage info. "No coverage information was collected, exit without writing coverage information."

I've just installed nyc and it works for me. Now I got needed info within ".nyc_output", and if I run nyc report --reporter=lcov it will generate "coverage" folder like what istanbul should does.

@robwilkerson
robwilkerson commented Apr 14, 2016 edited

I'm stumped. I'm running istanbul@0.4.3 and mocha@2.4.5. My mocha.opts file includes the --harmony_destructuring flag. Running npm test executes this script:

sudo -u www-data NODE_ENV=test mocha

The tests execute exactly as expected. When I execute the coverage script, though, I always get an unknown option error:

error: unknown option `--harmony_destructuring'

I've tried each of the following:

$ NODE_ENV=test node --harmony_destructuring node_modules/istanbul/lib/cli.js cover node_modules/.bin/_mocha
$ NODE_ENV=test istanbul cover node_modules/.bin/_mocha

What am I doing wrong here?

@robwilkerson

Argh. Punitive headslap. I just realized someone had put the flag back in mocha.opts and the flag sent to mocha was throwing the error, of course. Apologies for any confusion. The following works just fine as long as the harmony flags are removed from mocha.opts:

NODE_ENV=test node --harmony_destructuring node_modules/istanbul/lib/cli.js cover node_modules/.bin/_mocha
@richvdh richvdh added a commit to matrix-org/matrix-js-sdk that referenced this issue Feb 15, 2017
@richvdh richvdh Make test coverage work again
* don't try to run npm under istanbul
* use _mocha instead of mocha because
  gotwarlost/istanbul#44 (comment)

Also:
* write a text report for better travis
* On jenkins, clear out old coverage reports so that we don't pick up last
  time's.
75d213f
@richvdh richvdh referenced this issue in matrix-org/matrix-js-sdk Feb 15, 2017
Merged

Make test coverage work again #368

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment