web-test-runner-plugin blocks code coverage

[basham]

I'm trying to get web-test-runner to run code coverage, as outlined in their guide.

When I include the @snowpack/web-test-runner-plugin in web-test-runner.config.js as recommended in the Snowpack Testing docs, it outputs this to the terminal, and the HTML report is empty:

View full coverage report at coverage/lcov-report/index.html

However, when I don't include the plugin in the config, it works as expected. It runs the coverage and the HTML report is built.

Code coverage: 100%
View full coverage report at coverage/lcov-report/index.html

I suspect the plugin is somehow blocking the coverage code from executing. How do we resolve this?

I have the latest versions (as of today) of these dependencies installed:

{
  "devDependencies": {
    "@esm-bundle/chai": "4.1.5",
    "@snowpack/web-test-runner-plugin": "0.1.3",
    "@web/test-runner": "0.9.5",
    "snowpack": "2.15.1"
  }
}

[jgroh9]

I'm experiencing this issue as well

[FredKSchott]

I'd love some help debugging this! In your node-modules directory, can you add a console.log to:

node_modules/@snowpack/web-test-runner-plugin/plugin.js

    async serve({request}) {
+    console.log('SERVE', request.url);
      if (request.url.startsWith('/__web-dev-server')) {

    transformImport({source}) {
+    console.log('TRANSFORM', source);

And then share the result? Specifically, I'm curious if these are catching any files that aren't coming from your codebase. If that's happening, the it could be that Snowpack is trying to serve some code-coverage helper files itself, which would explain the error.

[basham]

Here's the output, as requested, using my basham/rx-ui-patterns project. Let me know if you need anything else.

TRANSFORM /src/util/use/useValue.test.js?wtr-session-id=3af3cf52-c9e8-4c75-9ae5-7428e7dffa10
TRANSFORM /src/util/mode.test.js?wtr-session-id=9ae8d6c1-e709-4a10-b185-b2fb6c1c8655
SERVE /util/use/useValue.test.js
SERVE /util/mode.test.js
TRANSFORM /__web-dev-server__web-socket.js
TRANSFORM ../../web_modules/@esm-bundle/chai.js
TRANSFORM ../web_modules/@esm-bundle/chai.js
TRANSFORM ../../web_modules/rxjs.js
TRANSFORM ./mode.js
TRANSFORM ./useValue.js
SERVE /web_modules/@esm-bundle/chai.js
SERVE /web_modules/rxjs.js
SERVE /util/mode.js
SERVE /util/use/useValue.js
TRANSFORM ../../web_modules/rxjs.js
TRANSFORM ./common/map-f121bde3.js
TRANSFORM ./object.js
TRANSFORM ../../web_modules/rxjs/operators.js
TRANSFORM ./common/map-f121bde3.js
TRANSFORM ./common/zip-85aacb30.js
TRANSFORM ./common/zip-85aacb30.js
SERVE /util/object.js
SERVE /web_modules/common/map-f121bde3.js
SERVE /web_modules/common/zip-85aacb30.js
SERVE /web_modules/rxjs/operators.js
TRANSFORM ./map-f121bde3.js
TRANSFORM ../common/map-f121bde3.js
TRANSFORM ../common/map-f121bde3.js
TRANSFORM ../common/zip-85aacb30.js
TRANSFORM ../common/zip-85aacb30.js

[basham]

To clarify, there are only two test files in this project right now.

/src/util/use/useValue.test.js
/src/util/mode.test.js

https://github.com/basham/rx-ui-patterns/blob/master/src/util/use/useValue.test.js
https://github.com/basham/rx-ui-patterns/blob/master/src/util/mode.test.js

[FredKSchott]

Hmm, I don't see anything surprising/problematic in that code. It looks like we're only serving your source files and test files, but not any other files. @LarsDenBakker any idea what may be going on here?

[LarsDenBakker]

Thanks for pinging me. It looks like one of the libraries we're using for test coverage parsing (https://github.com/istanbuljs/v8-to-istanbul) is returning empty results when using the plugin - possibly because the paths are pointing to dist. I'm looking into how we can resolve this.

[LarsDenBakker]

I've fixed this in @web/test-runner 0.9.7 . I also added web_modules to the default list of excluded folders for test coverage.

However there is still a few problems when doing test coverage with snowpack.

1. The files built with snowpack don't have any sourcemaps, so the generated report from istanbul can't map back to the original source content. (though this doesn't affect the actual coverage % calculation).

2. There are .proxy.js files that show up in the coverage report. I think that if we add sourcemaps istanbul will cascade them back the original files.

3. Test files are included in the test coverage. We exclude test files in the test runner by default, but the snowpack plugin rewrites the paths to the testfiles so there is a mismatch. Users can add an ignore pattern for this in the config, but it would be ideal if they didn't have to. How does this work with the dev server? Does it rewrite the paths to entrypoint files as well?

[basham]

I upgraded to @web/test-runner@0.9.7. I can confirm that coverage percentage outputs to the terminal. But yes, the report is broken when inspecting individual files. For example:

Unable to lookup source: /Volumes/Basham HD/GitHub/rx-ui-patterns/util/mode.js (ENOENT: no such file or directory, open '/Volumes/Basham HD/GitHub/rx-ui-patterns/util/mode.js')

The path is missing the src/ folder. The correct file path is:

/Volumes/Basham HD/GitHub/rx-ui-patterns/src/util/mode.js

[FredKSchott]

Great, thanks @LarsDenBakker! We’re tracking the remaining work to get sourcemaps working in all cases in #1213