ng test --code-coverage gives wrong results and do not include all files

[zielinskikamil]

Bug Report or Feature Request (mark with an x)

- [x] bug report
- [ ] feature request

Area

- [x] devkit
- [ ] schematics

Versions

node v8.9.4
npm 5.6.0
macOS Sierra

Repro steps

ng new CoverageTest
cd CoverageTest
ng g component componentA
ng g component componentB
// add dumy public method to componentb.ts
/*
 Additionally create some file and put there some simple class with one method. eg.
class ServiceA {
    public getHello(): string {
        return 'hello';
    }
}
*/

// now generate code coverage
ng test --code-coverage --watch false
// open coverage/index.html

Description

Code coverage generates a report but its misleading:

1. It doesn't include ServiceA coverage
2. Despite there are just two methods in componentb, coverage is eqal to 80%. If I preview covered area it covered untested file as well!
3. After editing test.ts (replace
   const context = require.context('./', true, /\.spec\.ts$/);
   with
   const context = require.context('./', true, /\/app\/.*\.ts$/);
   and reruning ng test --code-coverage --watch false
   ServiceA is visible in report but has some unrealistic values (Statement Coverage at 80%, Branches at 100%, Lines 83%)

Desired functionality

Ad 1. Coverage that does not include all files is misleading. How do I know which files are covered in unit tests?
Ad 2. How it happened that componentB is covered in 80% and newly created method is not covered in view? I suspect that mechanism that loads JS makes such faults positive scenario.
Ad 3. Why ServiceA shows as being covered despite there were no test for it?

All these 3 things look like a bug. Can you guys take a look at it?

[jeanpaulattard]

I have also encountered this issue.

I've starting writing tests for an application which has a considerable amount of files, and when i've opened the coverage report, to my surprise it only contained the files under test. (in previous projects working with boiler plate code, it always displayed all the files).

Is there a workaround for this? Do I have a missing config, or is it an issue in angular cli?

[sumeet-singh04]

Any updates on this issue ? I am able to exclude files using the angular.json, but not sure how to add them.

[LMFinney]

@sumeet-singh04 I just tried this approach, and it seemed to work without slowing my tests down appreciably. Does anyone know a downside to do this?

[Dinosys]

This seems like a quite a severe bug

Current workaround which seems to do the trick:

change src/test.ts:

const context = require.context('./', true, /\.spec\.ts$/);
to
const context = require.context('./app/', true, /\.ts$/);

in src/tsconfig.spec.json add
"**/*.ts"
to the include array

This will test all your source files for coverage, including those without a spec file.
If you then want to exclude files that you don't need coverage on you can add them to codeCoverageExclude array in angular.json in the test configuration under options.
We don't feel the need to check app.module for coverage, as well as in-memory-service.ts which is used for mocking data and not included in a production build

"test": {
          "builder": "@angular-devkit/build-angular:karma",
          "options": {
            "main": "src/test.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.spec.json",
            "karmaConfig": "src/karma.conf.js",
            "styles": [
              "src/styles.css"
            ],
            "scripts": [],
            "assets": [
              "src/favicon.ico",
              "src/assets"
            ],
            "sourceMap": true,
            "codeCoverage": true,
            "codeCoverageExclude": [
              "src/app/in-memory-service.ts",
              "src/app/app.module.ts"
            ]
          }
        }

[gabmorais]

Any updates on this issue ?

[demisx]

I've tried @Dinosys's workaround in Angular 7.2.10, but the coverage is never shown on ng test --no-watch. It compiles but then hangs and exits with this message 😞:

26 03 2019 18:06:47.791:WARN [HeadlessChrome 73.0.3683 (Mac OS X 10.14.3)]: Disconnected (0 times), because no message in 30000 ms.
HeadlessChrome 73.0.3683 (Mac OS X 10.14.3) ERROR
  Disconnected, because no message in 30000 ms.

[CraigComeOnThisNameCantBeTaken]

I am unable to get any workarounds working

[danieldanielecki]

It seems to be this issue hase been recently mentioned in this Medium post.

[cmacdonnacha]

To include all files use this method: https://stackoverflow.com/questions/52102998/angular-cli-get-coverage-report-to-include-all-sources