Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent Coverage Report by NYC #537

Closed
CamelKing opened this issue Mar 23, 2017 · 33 comments
Closed

Inconsistent Coverage Report by NYC #537

CamelKing opened this issue Mar 23, 2017 · 33 comments

Comments

@CamelKing
Copy link

CamelKing commented Mar 23, 2017

Expected Behavior

I am expecting consistent coverage report from test to test, unless there is a code change.

Observed Behavior

Without changing any of the code, source or test, running nyc consecutively gives different coverage % in the report.

Screen shots attached in gist.

Forensic Information

Operating System: Mac OSX Sierra.
Environment Information: NodeJS, Typescript.

** output.txt. @ https://gist.github.com/CamelKing/ca471857202851d81f57e0a9473ec7fd#file-output-txt

** screen shot 1 @ https://gist.github.com/CamelKing/ca471857202851d81f57e0a9473ec7fd#file-first-run

** screen shot 2 @ https://gist.github.com/CamelKing/ca471857202851d81f57e0a9473ec7fd#file-second-run-1sec-later

@CamelKing
Copy link
Author

CamelKing commented Mar 23, 2017

Upon further experiment, I noticed from time to time I see messages such as:

File [xxx.ts] ignored, nothing could be mapped.

Whenever this message appears (sometimes a few lines, each for one module), then I won't get 100% coverage for Branch %.

This seems to be random and intermittent, I can only reproduce it by repeatedly running the test and coverage script.

I am using npm script as followings:

"PORT=8000 nodemon --ext ts --watch ./ --exec 'nyc --reporter=text --reporter=html mocha -r ts-node/register ./test/**/*.spec.ts --recursive --reporter spec --slow 0' --delay 1"

Hopes this help.

@peterharding
Copy link

peterharding commented Mar 27, 2017

I am seeing the same (branches sometimes 60%, sometimes 100%).

@bcoe
Copy link
Member

bcoe commented Mar 27, 2017

@peterharding @CamelKing do me a favor and try running _mocha instead of mocha; my guess is that this is related to a subprocess exiting before coverage is written.

@peterharding are you also using nodemon?

@peterharding
Copy link

peterharding commented Mar 27, 2017

@bcoe Same problem with _mocha, I did a git diff on my lcov.info files and got:

diff --git a/lcov_30.info b/lcov_30.info
index 95c4dea..d9f64e2 100644
--- a/lcov_30.info
+++ b/lcov_30.info
@@ -250,15 +250,15 @@ DA:130,0
 LF:49
 LH:28
 BRDA:78,0,0,3
-BRDA:78,0,1,NaN
-BRDA:78,0,2,NaN
-BRDA:78,0,3,NaN
+BRDA:78,0,1,1
+BRDA:78,0,2,0
+BRDA:78,0,3,0
 BRDA:107,1,0,0
-BRDA:107,1,1,NaN
+BRDA:107,1,1,0
 BRDA:112,2,0,0
 BRDA:112,2,1,0
 BRF:8
-BRH:1
+BRH:2
 end_of_record

The commands I am running (package.json) are:

"test": "_mocha -r ts-node/register './**/*.spec.ts'",
"cover": "nyc npm test",

@bcoe
Copy link
Member

bcoe commented Mar 27, 2017

@peterharding interesting, that NaN definitely seems like it could be the culprit;

are you (or are you @CamelKing) able to share the project having issues?

@bcoe
Copy link
Member

bcoe commented Mar 28, 2017

@peterharding @CamelKing I find it interesting that typescript node register is used in both cases, this might be a good starting point for the investigation.

@JaKXz
Copy link
Member

JaKXz commented Mar 28, 2017

@CamelKing @peterharding I usually avoid using the _mocha executable with nyc. I've made a couple comments elsewhere about why that is... but since I don't have the time to take a quick look, I would recommend following this guide for nyc + mocha + typescript which I will be putting on http://istanbul.js.org when I have cycles.

EDIT: TS tutorial here: https://istanbul.js.org/docs/tutorials/typescript/

@CamelKing
Copy link
Author

CamelKing commented Mar 28, 2017

@bcoe I have tried mocha and _mocha, with or without nodemon, or npm watch. Same.

Initially I thought may be I have a cyclical requires amongst my files, spent some time to clear them, still no change. As the project gain more files, it seem the chances of getting the 'nothing to mapped' problem is higher. I now have to run multiple times before I managed to get a clean report.

My repo is at https://github.com/CamelKing/alpha.

The next thing I am gonna try is to pre-transpile all ts codes into js files and run the test without ts-node.

@CamelKing
Copy link
Author

@JaKXz thanks for the suggestion, I just tried the guide, but having the same problem.

@JaKXz
Copy link
Member

JaKXz commented Mar 28, 2017

@CamelKing in my PR I have gotten coverage working and added some of my opinions about package publishing while trying to maintain compatibility with your workflow. :) let me know what you think you'd like to see added or changed but hopefully this has streamlined and simplified things for you.

There are some important things to note:

  • if you want to get coverage reports on every watch run, you can see the discussion on useage with mocha --watch #310 and Coverage when using mocha --watch #200. IMHO it's not useful in each watch run, but to just go through the TDD workflow with npm test knowing what coverage you had before and increasing that number to match your thresholds, then using npm run coverage after your session to make sure you've hit it.
  • if you really want watch coverage, I believe you can add mochista but we do not own that tool [as cool as it looks] so I can't promise it will work.

Couple things I learned:

  • ts-node throws errors when .d.ts files are required.
  • the all: true behaviour requires all files that match the list of extensions, even if they are outside the include list, which is why I had to add "exclude": ["**/*.d.ts"]. This sounds like a bug in nyc or istanbuljs somewhere @bcoe and I think should be an easy-ish fix? I can investigate this later in a separate issue 😄

@CamelKing
Copy link
Author

@JaKXz , thanks for the PR, it sure helps clean up the tasks quite a fair bit. I have merged it into the master.

However, I have been testing it, and it still does produce the long list of 'nothing could be mapped' messages. Again this is not consistent, as in if I will get different results for every npm run coverage.

I have turned the npm loglevel to silly. The following is what I see.

If I get the 'nothing to be mapped' messages:

npm verb lifecycle alpha@0.0.1~coverage: unsafe-perm in lifecycle true
npm verb lifecycle alpha@0.0.1~coverage: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/camel/orange/alpha/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
npm verb lifecycle alpha@0.0.1~coverage: CWD: /Users/camel/orange/alpha
npm sill lifecycle alpha@0.0.1~coverage: Args: [ '-c', 'nyc mocha' ]
npm sill lifecycle alpha@0.0.1~coverage: Returned: code: 1 signal: null
npm info lifecycle alpha@0.0.1~coverage: Failed to exec coverage script
npm verb stack Error: alpha@0.0.1 coverage: nyc mocha
npm verb stack Exit status 1
npm verb stack at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:279:16)
npm verb stack at emitTwo (events.js:106:13)
npm verb stack at EventEmitter.emit (events.js:192:7)
npm verb stack at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:40:14)
npm verb stack at emitTwo (events.js:106:13)
npm verb stack at ChildProcess.emit (events.js:192:7)
npm verb stack at maybeClose (internal/child_process.js:890:16)
npm verb stack at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
npm verb pkgid alpha@0.0.1
npm verb cwd /Users/camel/orange/alpha
npm ERR! Darwin 16.4.0
npm ERR! argv "/usr/local/Cellar/node/7.6.0/bin/node" "/usr/local/bin/npm" "run" "coverage"
npm ERR! node v7.6.0
npm ERR! npm v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! alpha@0.0.1 coverage: nyc mocha
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the alpha@0.0.1 coverage script 'nyc mocha'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the alpha package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! nyc mocha
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs alpha
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls alpha
npm ERR! There is likely additional logging output above.
npm verb exit [ 1, true ]
npm ERR! Please include the following file with any support request:
npm ERR! /Users/camel/orange/alpha/npm-debug.log

On cases where it went thru successfully:

npm verb lifecycle alpha@0.0.1~coverage: unsafe-perm in lifecycle true
npm verb lifecycle alpha@0.0.1~coverage: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/camel/orange/alpha/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
npm verb lifecycle alpha@0.0.1~coverage: CWD: /Users/camel/orange/alpha
npm sill lifecycle alpha@0.0.1~coverage: Args: [ '-c', 'nyc mocha' ]
npm sill lifecycle alpha@0.0.1~coverage: Returned: code: 0 signal: null
npm info lifecycle alpha@0.0.1~postcoverage: alpha@0.0.1
npm sill lifecycle alpha@0.0.1~postcoverage: no script for postcoverage, continuing
npm verb exit [ 0, true ]
npm info ok

Both the above having same source code, same config.

@JaKXz
Copy link
Member

JaKXz commented Mar 29, 2017

FWIW, I would try turning it off and on again:

rm -rf node_modules
npm run clean
npm cache clean
npm install -g npm@latest
npm install

Also node v7.7.4 is out which may contain fixes. Otherwise, honestly I'm not sure why there are intermittent failures... sorry: it works on my machine so I'm in the dark and can only suggest rudimentary steps now.

@CamelKing
Copy link
Author

CamelKing commented Mar 29, 2017

@JaKXz I did two tests today.

  • make a new user on my mac today with empty settings, install the whole project from scratch, and it still produce the same result.

  • change the workflow. I now make tsc compile all .ts files (test included) into the lib directory, and then run nyc against all .js files from there. It works, no error report at all. The report is, to my surprise, in accordance to my original .ts files. Must be the magic of the sourcemap (which is not inline in my case).

For now, option 2 above will work for me, although it is a bit messy.

It seems the problem is probably with ts-node, or the code interacting with ts-node?

I made a tag in my repo, nyc_issue, in case any of you guys wanna take a look at the situation and may be even figure out a fix :)

Btw, I am using an iMac 27" (late 2012 model), macOS Sierra 10.12.3. Not sure how much difference that would make.

Once again, thanks for your help. @bcoe @peterharding too. Cheers.

@RecuencoJones
Copy link

RecuencoJones commented Apr 11, 2017

I am experiencing the same issue with the following:

Dependencies:

  • nyc: 10.2.0
  • mocha: 3.1.2
  • typescript: 2.1.5
  • ts-node: 3.0.2

Env:

  • npm: 4.0.5
  • node: 7.4.0
  • os: macOS 10.12.3 (16D32)

npm scripts:

  ...
  "scripts": {
    ...
    "test:unit": "mocha -r ts-node/register test/specs/unit/**/*.spec.ts",
    "test:coverage": "nyc npm run test:unit"
  },
  ...

.nycrc

{
  "include": [
    "app/**/*.ts"
  ],
  "exclude": [
    "app/Index.ts"
  ],
  "require": [
    "ts-node/register"
  ],
  "extension": [
    ".ts"
  ],
  "reporter": [
    "json",
    "lcov",
    "text-summary"
  ],
  "all": true,
  "report-dir": "./test/results/unit/coverage"
}

In our case the problem got solved downgrading to nyc v9.1.0, consistent 100% of the times.

The project source is a single TypeScript file and the error comes in the branching of an if statement, even though both branches are covered by unit tests.

Edit: Added npm scripts and .nycrc config

@CamelKing
Copy link
Author

Yes, in fact as the number of modules grow in my project (now >100), it is more likely to happen. I typically run the coverage tool a few times until I get one without the error message, and this is taking up a lot of time ... my worst fear is I am not 100% sure the coverage report is accurate ...

@bcoe
Copy link
Member

bcoe commented Apr 16, 2017

@CamelKing my hunch is that we're bumping into something similar to this issue, basically I think there's a chance that a file might be getting instrumented twice.

Can you provide me with a fairly minimal branch to run, that exhibits the flappy behavior?

@bcoe
Copy link
Member

bcoe commented Apr 16, 2017

@CamelKing I've made some headway, it actually seems to be related to caching, and SourceMaps; will report back soon.

@bcoe
Copy link
Member

bcoe commented Apr 16, 2017

@CamelKing, do me a favor and try reproducing with --cache=false; I believe the issue is that files fail to map on the second run, when source is loaded from cache.

Would love help digging into this.

@bcoe
Copy link
Member

bcoe commented Apr 19, 2017

@CamelKing could you please try:

npm i nyc@next

I think I've hunted down the root cause of your issues (#556) but it's a somewhat frightening change, so I'd love some help QAing and validating the work.

@CamelKing
Copy link
Author

CamelKing commented Apr 19, 2017

Hi, @bcoe sorry for the late response.

I have install nyc@next and tried running it for 10 times. While the error message does not occur, I notice the followings two outcomes at random, with no change to source code:

Statements : 86.76% ( 1304/1503 )
Branches : 90.62% ( 570/629 )
Functions : 92.96% ( 198/213 )
Lines : 87.31% ( 1273/1458 )

Statements : 95.74% ( 1304/1362 )
Branches : 91.2% ( 570/625 )
Functions : 93.84% ( 198/211 )
Lines : 95.99% ( 1174/1223 )

It seems while the error message is suppressed, the impact is still there. The one is lower coverage percentage is the same as the one with error message before.

I then tried running npm with silly log level, notices the followings:

npm verb lifecycle alpha@0.1.0~cover: unsafe-perm in lifecycle true
npm verb lifecycle alpha@0.1.0~cover: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/camel/orange/alpha/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
npm verb lifecycle alpha@0.1.0~cover: CWD: /Users/camel/orange/alpha
npm sill lifecycle alpha@0.1.0~cover: Args: [ '-c', 'nyc mocha --opts mocha.opts' ]
npm sill lifecycle alpha@0.1.0~cover: Returned: code: 1  signal: null
npm info lifecycle alpha@0.1.0~cover: Failed to exec cover script
npm verb stack Error: alpha@0.1.0 cover: `nyc mocha --opts mocha.opts`
npm verb stack Exit status 1
npm verb stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:279:16)
npm verb stack     at emitTwo (events.js:106:13)
npm verb stack     at EventEmitter.emit (events.js:194:7)
npm verb stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:40:14)
npm verb stack     at emitTwo (events.js:106:13)
npm verb stack     at ChildProcess.emit (events.js:194:7)
npm verb stack     at maybeClose (internal/child_process.js:899:16)
npm verb stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
npm verb pkgid alpha@0.1.0
npm verb cwd /Users/camel/orange/alpha
npm ERR! Darwin 16.5.0
npm ERR! argv "/usr/local/Cellar/node/7.7.4/bin/node" "/usr/local/bin/npm" "run" "cover" "--loglevel" "silly"
npm ERR! node v7.7.4
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! alpha@0.1.0 cover: `nyc mocha --opts mocha.opts`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the alpha@0.1.0 cover script 'nyc mocha --opts mocha.opts'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the alpha package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     nyc mocha --opts mocha.opts
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs alpha
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls alpha
npm ERR! There is likely additional logging output above.
npm verb exit [ 1, true ]

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/camel/orange/alpha/npm-debug.log

running mocha on it's own with the same config would yield the followings:

npm verb lifecycle alpha@0.1.0~mocha: unsafe-perm in lifecycle true
npm verb lifecycle alpha@0.1.0~mocha: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/camel/orange/alpha/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
npm verb lifecycle alpha@0.1.0~mocha: CWD: /Users/camel/orange/alpha
npm sill lifecycle alpha@0.1.0~mocha: Args: [ '-c', 'mocha --opts mocha.opts' ]
npm sill lifecycle alpha@0.1.0~mocha: Returned: code: 0  signal: null
npm info lifecycle alpha@0.1.0~postmocha: alpha@0.1.0
npm sill lifecycle alpha@0.1.0~postmocha: no script for postmocha, continuing
npm verb exit [ 0, true ]
npm info ok

So the problem only arises when calling mocha from nyc.

@CamelKing
Copy link
Author

@bcoe If you need a smaller branch to test, I have created a tag: nyc issue. This is the state of my project when I first reported the issue.

@bcoe
Copy link
Member

bcoe commented Apr 19, 2017

@CamelKing I believe I've solved one other issue, could you please try npm cache clear; npm i nyc@next again?

@CamelKing
Copy link
Author

@bcoe test run 10 times and

  1. No 'nothing to map' error message
  2. Consistent coverage report

I will continue to run this for the next week or so and report back if I encountered any issue.

I did try to run it with silly npm log level, and observed the followings:-

npm verb lifecycle alpha@0.1.0~cover: unsafe-perm in lifecycle true
npm verb lifecycle alpha@0.1.0~cover: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/camel/orange/alpha/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
npm verb lifecycle alpha@0.1.0~cover: CWD: /Users/camel/orange/alpha
npm sill lifecycle alpha@0.1.0~cover: Args: [ '-c', 'nyc mocha --opts mocha.opts' ]
npm sill lifecycle alpha@0.1.0~cover: Returned: code: 1  signal: null
npm info lifecycle alpha@0.1.0~cover: Failed to exec cover script
npm verb stack Error: alpha@0.1.0 cover: `nyc mocha --opts mocha.opts`
npm verb stack Exit status 1
npm verb stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:279:16)
npm verb stack     at emitTwo (events.js:106:13)
npm verb stack     at EventEmitter.emit (events.js:194:7)
npm verb stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:40:14)
npm verb stack     at emitTwo (events.js:106:13)
npm verb stack     at ChildProcess.emit (events.js:194:7)
npm verb stack     at maybeClose (internal/child_process.js:899:16)
npm verb stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
npm verb pkgid alpha@0.1.0
npm verb cwd /Users/camel/orange/alpha
npm ERR! Darwin 16.5.0
npm ERR! argv "/usr/local/Cellar/node/7.7.4/bin/node" "/usr/local/bin/npm" "run" "cover" "--loglevel" "silly"
npm ERR! node v7.7.4
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! alpha@0.1.0 cover: `nyc mocha --opts mocha.opts`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the alpha@0.1.0 cover script 'nyc mocha --opts mocha.opts'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the alpha package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     nyc mocha --opts mocha.opts
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs alpha
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls alpha
npm ERR! There is likely additional logging output above.
npm verb exit [ 1, true ]

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/camel/orange/alpha/npm-debug.log

Not sure if this mean anything, but as far as I can tell, things seems running fine and I am ignoring this error message (by turning it off, LOL).

@bcoe
Copy link
Member

bcoe commented Apr 20, 2017

@CamelKing I believe the error output is that you're below your coverage thresholds; which causes nyc to exit with a 1.

@CamelKing
Copy link
Author

@bcoe ah ... got it. Thanks :)

@bcoe
Copy link
Member

bcoe commented Apr 20, 2017

@CamelKing FYI, make sure you install the latest nyc@next; fixed a bug that would cause things to fail if you had no node_modules/.cache folder.

@bcoe bcoe closed this as completed in ff73b18 Apr 20, 2017
@abrenneke
Copy link

This is still happening for me in 11.8.0 if the --all flag is passed. I am using ava which is transpiling both the test files and the source files using babel-register.

Removing --all makes the numbers consistent between runs (but I would like to specify it so that untested files are included)

@canvaspixels
Copy link

@SneakyMax you legend, this is what i've been after for a while.

@ralphholzmann
Copy link

I am experiencing this still as well with 11.8.0 and mocha@5.2.0. The --all flag seems to be a culprit.

@niedzielski
Copy link

Same issue on nyc v12.0.2 and QUnit v2.6.1. The coverage rates and uncovered line #s vary across runs when all is enabled.

@mohitkhanna
Copy link

Same thing here. With latest nyc as of today the same behavior is being seen. I've tried cache:false and all:true and other combinations. It's not working.

@coreyfarrell
Copy link
Member

@ralphholzmann @niedzielski @mohitkhanna Please open new issues with details including a link to an example repo showing the problem. These details will make it easier to find the specific cause of your issue so it can be dealt with.

@ralphholzmann
Copy link

@coreyfarrell thanks -- my issue is here #849 with a repo that reproduces the bug

wmfgerrit pushed a commit to wikimedia/mediawiki-extensions-Popups that referenced this issue Jun 19, 2018
Replace Istanbul with nyc, Istanbul's CLI. nyc appears to have some
bugs that this patch works around:

  - When all files in the project are considered, not just those
    imported via tests, the coverage rates and line numbers vary between
    runs. This patch disables the `all` option for now and points to the
    bug:
    istanbuljs/nyc#537 (comment).

  - Source map line numbers appear to be incorrect except when `all` is
    enabled and working correctly (see previous bullet).

  - `sourceMap` must be disabled to avoid ENAMETOOLONG errors when nyc
    tries to include them as encoded strings. The patch disables the
    setting and points to: istanbuljs/nyc#847.
    Using babel-plugin-istanbul and source-map-support appears to have
    no effect (the former in tests/node-qunit/run.js and .babelrc).

  - CI fails with
    `Error: EACCES: permission denied, mkdir '/nonexistent'`. Specify
    `SPAWN_WRAP_SHIM_ROOT` instead of constructing a subdirectory from a
    nonexistent home directory.

Bug: T196952
Bug: T193519
Change-Id: Idf2e3accd4a6277cbef91c1156fcd206c9e7d882
jthomerson pushed a commit to silvermine/generator-typescript that referenced this issue Jan 10, 2019
Running tests multiple times in a row was producing inconsistent
coverage reports. About half of the time the coverage report was 50%
coverage and the other half of the time the coverage report was at 75%.
In searching the nyc Github repo, I found a Github issue that described
this problem. The issue had a comment by one of the maintainers [1] that
linked to a blog post he'd written that describes a recommended set of
configuration for NYC, Mocha, and TypeScript [2]. There, I noticed that
the example nycrc.json file had configuration to require
'ts-node/register', as mocha does in this repo now. He does not explain
why it's required, and why it should be done in NYC instead of Mocha,
but adding that to our nycrc.json fixed the inconsistent code coverage
reports and they now correctly and consistently report 75%.

[1]: istanbuljs/nyc#537 (comment)
[2]: https://azimi.me/2016/09/30/nyc-mocha-typescript.1.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests