Only show 100%, not listing the coverage of each file. #38

RiverLi · 2016-11-03T06:09:54Z

Followed the example here but got report only shows 100%. It is not listing the file name and the corresponding coverage of each file.
{
"suites": ["path/myTestSuite.html"],
"verbose": true,
"plugins": {
"local": {
"browsers": ["chrome"]
},
"istanbul": {
"dir": "./coverage",
"reporters": ["text-summary", "lcov"],
"include": [
"path/some.html",
"path/some.js",
"path/some.html",
"path/some.js",
"path/some.html",
"path/some.js"
],
"exclude": []
}
}
}

Would we list the coverage of each file? Like the unit test coverage of java.
First level, it shows the percentage of a java Component
Second level, it shows the percentage of a namespace.
Third level, it shows the percentage of a java file.

klarkc · 2016-12-20T20:23:19Z

Same here, with web-component-tester-istanbul@0.10.0 and web-component-tester@6.0.0-prerelease.4:

"istanbul": {
      "dir": "coverage",
      "reporters": ["text-summary", "lcov"],
      "include": [
        "**/*.js",
        "**/*.html"
      ]
    }

ctreatma · 2016-12-21T16:52:26Z

In the coverage output 100% (0/0), the part in parentheses is the <covered lines of code>/<total lines of code>, so the report indicates 100% coverage and omits file-specific coverage because istanbul thinks there's no code/files to measure.

It appears that the web-component-tester-istanbul middleware is successfully added to the app stack when the prepare:webserver hook fires, but the middleware is never actually used for any requests, so code is never instrumented for coverage. I suspect that, as a result of the move to polyserve for wct, the instrumenting middleware is injected too late, so all requests are handled & completed by earlier middleware/routing. However, I don't have enough bandwidth to dig into this further at the moment.

I can't tell if this issue is related to PolymerLabs/polyserve#138; at this point, we need some guidance from the wct maintainers as to how (or if) they expect middleware like web-component-tester-istanbul to inject behavior into the wct request handling pipeline after the migration to polyserve.

sam1463 · 2017-01-16T19:18:44Z

Any update here? Looks to me like @ctreatma is correct about the middleware not being used after being added and that we might need more information from the wct team to address this. Maybe open an issue on the wct repo to see what they have to say

elf-pavlik · 2017-02-26T16:54:12Z

Does this merged PR help in any way? Polymer/web-component-tester#445

jhpedemonte · 2017-03-16T20:15:55Z

@elf-pavlik I tested with web-component-tester#6.0.0-prerelease.7, which includes that fix. Fixed the express.use error, but not this issue.

t2ym · 2017-03-18T12:54:27Z

Root Cause:

The root cause is polyserve, which handles all the HTTP requests and makes extra middleware injections ineffective.

My patches

I created patched versions of web-component-tester, polyserve, and web-component-tester-istanbul to make the plugin middleware effective. I also changed web-component-tester-istanbul to handle responses as a response stream transformer instead of fetching local files. Unfortunately, the patched versions are architecturally incompatible with the current versions.

I am wondering how to merge the patches into the official versions. Any suggestions on my approach?

In addition, I patched web-component-tester to fix a timeout issue with Sauce Labs, caused by large coverage data via HTTP POST seemingly rejected by the squid proxy in Sauce Labs. The large coverage data are chopped in the browser into stringified JSON fragments not larger than 64KB and transparently combined at the server.

Forks with the patches:

https://github.com/t2ym/web-component-tester/tree/wct6-plugin
https://github.com/t2ym/polyserve/tree/istanbul
https://github.com/t2ym/web-component-tester-istanbul/tree/wct6

npm packages with the patches.

"web-component-tester": "https://github.com/t2ym/web-component-tester#wct6-plugin",
"web-component-tester-istanbul": "https://github.com/t2ym/web-component-tester-istanbul#0.10.1-wct6.12",

bower package with the patches:

"web-component-tester": "t2ym/web-component-tester#wct6-plugin"

Example project with patched `polymer-cli` which uses the patched wct6 and the plugin

https://github.com/t2ym/i18n-element

srk289 · 2017-04-06T00:01:43Z

@t2ym I tried your forked repos but the issue still remains. It always shows 100% (0/0). Has anyone else had any success?

t2ym · 2017-04-06T00:57:15Z

@srk289 Thank you for trying my forks. Can you please try this successful example with the patched ./node_modules/.bin/wct? As my fork of web-component-tester-istanbul has received 2 GitHub stars, I believe others should have succeeded in coverage.

Example successful coverage with the patched modules

$ node --version
v6.9.2
$ git clone https://github.com/t2ym/i18n-element.git
$ cd i18n-element
$ npm install
$ bower install
$ ./node_modules/.bin/wct
Received undefined
Creating Sauce Connect tunnel
Sauce Connect log: /tmp/wct11736-9226-v1w7q3.h4il6gk3xr/sc.log
Sauce tunnel active: b373cf94-4c35-4cd6-a6a8-0f189506822d
Windows 10 chrome        Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=0
OS X 10.12 safari 10     Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=3
OS X 10.11 safari 9      Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=5
Windows 8.1 IE 11        Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=4
Windows 10 microsoftedge 13 Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=2
Windows 10 chrome        Tests passed
Windows 10 firefox       Beginning tests via http://localhost:8081/components/i18n-element/generated-index.html?cli_browser_id=1
OS X 10.12 safari 10     Tests passed
Windows 8.1 IE 11        Tests passed
OS X 10.11 safari 9      Tests passed
Windows 10 microsoftedge 13 Tests passed
Windows 10 firefox       Tests passed
Test run ended with great success

=============================== Coverage summary ===============================
Statements   : 100% ( 175/175 )
Branches     : 91.49% ( 172/188 )
Functions    : 100% ( 25/25 )
Lines        : 100% ( 170/170 )
================================================================================

As per @srk289 's request, a simpler example with this change

$ git clone https://github.com/t2ym/paper-button.git
$ cd paper-button
$ npm install
$ bower install
$ ./node_modules/.bin/wct

srk289 · 2017-04-07T04:48:58Z

@t2ym Terrific. Thanks heaps for your help.

ghuntley · 2017-04-07T08:38:32Z

Actively watching this thread with anticipation. Thank-you for taking ownership of this 👍

marybeshaw · 2017-05-17T17:19:16Z

At this point, Polymer 2 has been released.
I haven't seen any movement on this thread, and I haven't seen any recent releases to web-component-tester-istanbul. It would be great to see these fixes incorporated into all of the tools.

@t2ym - Have polyserve and/or web-component-tester incorporated your bug fixes? I don't see any linked issues or PRs.

skye2k2 · 2017-05-17T18:20:39Z

web-component-tester#6.0.0 was released two days ago without having addressed a number of issues. Running it still exhibits this same behavior.

t2ym · 2017-05-17T23:31:02Z

@marybeshaw No. As described above, my patches surely work but trickily change the core design of the modules, which I am afraid Google would not accept. I am using my patched versions for my projects without troubles and currently it is too much for me to make tested high quality pull requests and persuade Google reviewers, who are very busy and seemingly putting a low priority on code coverage, to accept the design changes.

web-component-tester#6.0.0 has been merged into my forks with the patches.

t2ym · 2017-07-02T05:39:01Z

wct-istanbul

For easier maintenance and deployment, the patched plugin is published as wct-istanbul from wct-istanbul branch of my fork. The plugin requires the patched t2ym/web-component-tester#6.0.0-wct6-plugin.5 or later.

Forks with the patches:

https://github.com/t2ym/web-component-tester/tree/wct6-plugin-scoped
https://github.com/t2ym/polyserve/tree/istanbul
https://github.com/t2ym/web-component-tester-istanbul/tree/wct-istanbul
https://github.com/t2ym/polymer-cli/tree/wct6-plugin

npm packages with the patches.

"@t2ym/web-component-tester": "^6.0.0",
"@t2ym/polymer-cli": "^1.3.0",
"wct-istanbul": "^0.11.0",

bower package with the patches:

"web-component-tester": "t2ym/web-component-tester#^6.0.0"

t2ym · 2017-07-03T14:31:23Z

My fork wct-istanbul 0.12.0 now supports the latest IstanbulJS libraries as nyc v11

async/await syntax is supported by the IstanbulJS libraries.
nyc wct command is NOT supported

When `wct` moved from `serve-waterfall` to `polyserve`, it broke this plugin's usage of the `prepare:webserver` hook. With newer versions of `wct`, the app used by `wct` to serve component files has a wildcard request handler registered that prevents calling the instrumentation middleware. Version 6.4.0 of `wct` adds a new `define:webserver` hook so that plugins can inject middleware to be executed before the `polyserve` wildcard handler. This replaces the `prepare:webserver` handler with one for the new `define:webserver` event, allowing this plugin to instrument code on the fly again. Also fixes thedeeno#41

klarkc mentioned this issue Dec 20, 2016

Not getting any coverage? #12

Open

klarkc mentioned this issue Dec 20, 2016

Setup test coverage klarkc/streaming-list#28

Open

jehon mentioned this issue Feb 1, 2017

Code coverage: how to cover code in <script> tag in html files? karma-runner/karma-coverage#277

Open

t2ym mentioned this issue Jul 4, 2017

Create pull requests to merge the patches into the official web-component-tester t2ym/web-component-tester#1

Open

alessandro-aglietti mentioned this issue Aug 19, 2017

Code coverage w/ istanbul & wct growingabit/pwa-frontend#19

Closed

sam1463 mentioned this issue Jan 3, 2019

Question: Why plugin middleware is never getting called Polymer/tools#3134

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only show 100%, not listing the coverage of each file. #38

Only show 100%, not listing the coverage of each file. #38

RiverLi commented Nov 3, 2016

klarkc commented Dec 20, 2016 •

edited

ctreatma commented Dec 21, 2016

sam1463 commented Jan 16, 2017

elf-pavlik commented Feb 26, 2017

jhpedemonte commented Mar 16, 2017

t2ym commented Mar 18, 2017 •

edited

srk289 commented Apr 6, 2017

t2ym commented Apr 6, 2017 •

edited

srk289 commented Apr 7, 2017

ghuntley commented Apr 7, 2017

marybeshaw commented May 17, 2017

skye2k2 commented May 17, 2017

t2ym commented May 17, 2017

t2ym commented Jul 2, 2017 •

edited

t2ym commented Jul 3, 2017

Only show 100%, not listing the coverage of each file. #38

Only show 100%, not listing the coverage of each file. #38

Comments

RiverLi commented Nov 3, 2016

klarkc commented Dec 20, 2016 • edited

ctreatma commented Dec 21, 2016

sam1463 commented Jan 16, 2017

elf-pavlik commented Feb 26, 2017

jhpedemonte commented Mar 16, 2017

t2ym commented Mar 18, 2017 • edited

Root Cause:

My patches

Forks with the patches:

npm packages with the patches.

bower package with the patches:

Example project with patched polymer-cli which uses the patched wct6 and the plugin

srk289 commented Apr 6, 2017

t2ym commented Apr 6, 2017 • edited

Example successful coverage with the patched modules

As per @srk289 's request, a simpler example with this change

srk289 commented Apr 7, 2017

ghuntley commented Apr 7, 2017

marybeshaw commented May 17, 2017

skye2k2 commented May 17, 2017

t2ym commented May 17, 2017

t2ym commented Jul 2, 2017 • edited

wct-istanbul

Forks with the patches:

npm packages with the patches.

bower package with the patches:

t2ym commented Jul 3, 2017

My fork wct-istanbul 0.12.0 now supports the latest IstanbulJS libraries as nyc v11

klarkc commented Dec 20, 2016 •

edited

t2ym commented Mar 18, 2017 •

edited

Example project with patched `polymer-cli` which uses the patched wct6 and the plugin

t2ym commented Apr 6, 2017 •

edited

t2ym commented Jul 2, 2017 •

edited