Code coverage option is not returning correct coverage

[jfeng-salesforce]

Hi all!

I'm trying to get code coverage numbers for all LWC .js files (not just the ones with tests). I'm seeing some weird / probably unexpected behaviour with lwc-jest in versions 0.4.10, 0.4.11, and 0.4.12, and also cannot override the codeCoverageFrom attribute in jest.config.js in a way that would pick up the results. There seems to be some strange interaction between lwc-jest and basic jest which is causing this issue.

Setup:
I have a SFDX project with 2 LWC components:

• navbar (the only one with jest tests)
• footer

When I run:

lwc-jest --coverage

I'm hoping to see a result for all my LWC components (including those without tests).

-----------------------|----------|----------|----------|----------|-------------------|
File                   |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
 ----------------------|----------|----------|----------|----------|-------------------|
All files              |    blah  |     blah |     blah |     blah |                   |
 footer.html.compiled  |        0 |     blah |        0 |        0 |              blah |
 navBar.html.compiled  |    88.89 |       50 |      100 |    88.89 |                29 |
-----------------------|----------|----------|----------|----------|-------------------|

(where "blah" represents a made-up value)

What I actually see is:

In 0.4.10:

----------------------|----------|----------|----------|----------|-------------------|
File                  |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------------------|----------|----------|----------|----------|-------------------|
All files             |    88.89 |       50 |      100 |    88.89 |                   |
 navBar.html.compiled |    88.89 |       50 |      100 |    88.89 |                29 |
----------------------|----------|----------|----------|----------|-------------------|

In 0.4.11 and 0.4.12:

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |  Unknown |  Unknown |  Unknown |  Unknown |                   |
----------|----------|----------|----------|----------|-------------------|

If I try to override codeCoverageFrom at either the CLI (--codeCoverageFrom) or extending jest.config.js , then lwc-jest does not pick up ANY results from the /lwc/ directory.

This is configured with **/*.js, which picks up all other .js files - but no results from /lwc (not even the element with a test). This happens in 0.4.10 - 12.

--------------------------------------------------------------|----------|----------|----------|----------|-------------------|
File                                                          |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
--------------------------------------------------------------|----------|----------|----------|----------|-------------------|
All files                                                     |        0 |      100 |        0 |        0 |                   |
  jest.config.js                                              |        0 |      100 |        0 |        0 |... 16,17,18,19,22 |
--------------------------------------------------------------|----------|----------|----------|----------|-------------------|

Running with --verbose does not give any additional information or error messages :(

Is there some way to configure lwc-jest code coverage to return results for all LWC elements please? Thanks in advance!

[trevor-bliss]

Firstly, you should definitely be on version 0.4.11 or higher. Even if the output is unexpected for now. The bug fixed in 0.4.11 is needed for expected results.

Looking into your sample project after bumping to lwc-jest to version 0.4.12, I noticed both your LWC javascript files are essentially empty classes. Once you add more code into the classes you'll see it starting to behave as expected. This is why you get the Unknown entries when both classes have no logic in them. The coverage can't find any valid javascript files under force-app/main/default/lwc so has nothing to report.

I agree, however, that even with a skeleton class file for a component it should still show up on the coverage report as having 0% coverage. Let me dig in to this more and get back to you. For now though I don't think there's any bug at the lwc-jest configuration level.

[trevor-bliss]

This is fixed in Jest 24+. lwc-jest is still on version 23 for a couple reasons:

• LWC core requires changes to upgrade Jest that is only in the development branch
• Jest 24 is a major version bump with breaking changes we shouldn't ship mid-release

For the next major release we can upgrade Jest to fix this issue. Components with logic in their js classes should still report coverage as expected.

[jfeng-salesforce]

Both classes have the same pretty much empty .js file. If all three lwc-jest versions are built against Jest v23, why does the one with a test show up in 0.4.10, but not in 0.4.11 or 0.4.12? Why does the one file with a test show up as a ".html.compiled" file in 0.4.10 instead of a .js file (even though **/.js files are the only ones being added to the lwc-jest collectCoverageFrom classpath in https://github.com/salesforce/lwc-jest/blob/master/src/config.js for getCoveragePaths()? Is getCoveragePaths() actually returning the correct file filter? Is code coverage actually being compiled against .js instead of .html.compiled files?)

[trevor-bliss]

The bug present in 0.4.10 returns undefined for collectCoverageFrom, not all javascript files. It's fixed in 0.4.11. That's probably why you see the .html.compiled file in the coverage report in 0.4.10 but not later versions.

Coverage should be calculated against the javascript, not the html or html.compiled files.

I really don't think it's worth trying to figure out what exactly is wrong in 0.4.10 because we're on a higher version and remember, we get the LWC core version from this package so using an older version of lwc-jest means using an outdated version of LWC core in your tests. For current versions we know the fix is to upgrade Jest.

[jfeng-salesforce]

The only code-coverage-related difference between 0.4.10 and 0.4.12 is the return path for collectCoverageFrom, so I assume any underlying coverage issues in 0.4.10 will still be present in 0.4.12.

It sounds like what you're saying is --coverage is not completely functional in the current version of lwc-jest, and should be fixed in the next major release (with the upgrade to Jest 24+ compliance). Is that accurate?

[trevor-bliss]

Yes. Not fully functional for empty component classes. I have verified locally the empty classes are properly reported in coverage reports with Jest version 24.

[jfeng-salesforce]

Awesome, thank you so much Trevor!

[BatemanVO]

I'm seeing some strange behavior with the reported coverage of a component. I'm on lwc-jest version 0.5.1.

The lines that are highlighted red are marked as "uncovered", yet I don't see how that could be given the surrounding lines being covered. For example, how could this.lastJob = null not be covered if the lines before and after it are? Or, how could the catch(this.showError.bind(this)) be covered, but then showError is not covered? How could the variant property of the toast not be covered, but the title and message properties are?

I tried exposing the showError method with an @api annotation and calling it directly, but then it showed that the @api line was not covered, and the title attribute was not covered, although showError and variant now were.

I tried adding a comment, and that line now appeared as uncovered, so it seems as though the uncovered lines are always the same lines, regardless of what the file looks like.

Additionally, my file is only 61 lines long, but the "uncovered lines" section of the coverage output in the terminal shows line 80 as being uncovered.

Here's my jest.config.js:

[pmdartus]

@BatemanVO Can you share a repository with some reproduction steps?

[BatemanVO]

@pmdartus I've created a shareable repo. If you clone it down, run npm install, then run npm run test:unit, it should run lwc-jest --coverage && eslint force-app/main/default/lwc/. If you don't have eslint installed globally and it complains that the local version installed is not enough, you may need to do npm i -g eslint or just remove the eslint part of the package.json's test:unit script (the coverage problem appears with and without eslint).

Everything should run just fine if you deploy the classes in the classes folder, and you may need one custom label called "Alphabetize_Batch_Explanation" in the Org.

[trevor-bliss]

@BatemanVO I pulled down the repo you shared and I'm seeing coverage numbers on both Mac and Windows. Here's a screenshot of my results running lwc-jest --coverage on Windows:

Are these not the results you are expecting?

[BatemanVO]

@trevor-bliss That's the same result I get - except those uncovered line numbers don't make any sense. alphabetizeKeyDashboard.js is only 61 lines long, so line 80 couldn't be uncovered. Additionally, lines 45, 53, 54, and 57 are always uncovered, no matter how I change the file.

I opened up the coverage/lcov-report/index.html file, since it offers a UI to see what lines aren't covered, and that's the screenshot I included in my original comment. It shows properties of an object not being covered, but the other properties being covered, which doesn't make any sense.

I imagine that somehow alphabetizeKeyDashboard.js is being compiled into a larger file or something, but it's proving very difficult to figure out how I could increase coverage since what I am being shown isn't accurate.

[pmdartus]

Also pulled down the repository, at this commit. That being said running npm run test:unit, surfaces the following error:

Cannot find module 'c/AlphabetizeKeyDashboard' from 'alphabetize-key-dashboard.test.js'

I realized that the imported module is c/AlphabetizeKeyDashboard while the actual module name on the file system is c/alphabetizeKeyDashboard. I am running the project on a linux maching which may explain the case-sensitivity issue. @trevor-bliss do you run on the same issue on windows, the FS on windows is by default case-sensitive?

---

With that out of the issue out of the way here is a screenshot of the reported coverage which looks right to me.

@BatemanVO what is the OS of the machine?

[BatemanVO]

@pmdartus Changing the import statement to correctly reference the lowercased alphabetizeKeyBatch.js instead of AlphabetizeKeyBatch.js has properly fixed the coverage report for me. Strange that all the tests ran and, if I used debuggers or console.logs, all the code was in fact being reached and appropriately tested. Perhaps the returned coverage report was from an error file of some sort?

I am on Windows 10, Version 1803 (OS Build 17134.885).

[trevor-bliss]

This is really interesting. I also see coverage change on a Mac by changing the case. I've seen cases locally and on Jest issues about case-sensitivity, but they usually lead to hard failures on non-Mac environments, not just slightly different results like this.

Would love to know what the root cause is if anyone has time to dig into the sub-packages.