Spec reporter prints an incorrect summary

[devopsix]

Environment:

• WebdriverIO version: 6.1.22
• Mode: WDIO
• If WDIO Testrunner, running sync/async: sync
• Node.js version: 12.16.3
• NPM version: 6.14.4
• Browser name and version: Chrome 82
• Platform name and version: Windows 10
• Additional wdio packages used (if applicable): @wdio/spec reporter, @wdio/mocha-framework, wdio-chromedriver-service

Config of WebdriverIO
Will provide that if below information is not sufficient.

Describe the bug
Spec reporter prints an incorrect summary, e.g.: “Spec Files: 5 passed, 1 total (500% completed) in 00:01:02”

To Reproduce
Given you have "scripts": {"test": "wdio"} in your package JSON
and given the specs pattern matches multiple spec files
and given you use a multiremote instance through chromedriver (not sure if that matters, though)
then run npm test.

Expected behavior
Spec reporter should print a correct summary like: “Spec Files: 5 passed, 5 total (100% completed) in 00:01:02”

Log

(Some names redacted.)

 "spec" Reporter:
------------------------------------------------------------------
[MultiremoteBrowser (unknown) #0-0] Spec: XXXXXXXXXX.spec.ts
[MultiremoteBrowser (unknown) #0-0] Running: MultiremoteBrowser on (unknown)
[MultiremoteBrowser (unknown) #0-0]
[MultiremoteBrowser (unknown) #0-0] XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-0]
[MultiremoteBrowser (unknown) #0-0] 5 passing (4.9s)
[MultiremoteBrowser (unknown) #0-0]
[MultiremoteBrowser (unknown) #0-0] 
[MultiremoteBrowser (unknown) #0-0] 
------------------------------------------------------------------
[MultiremoteBrowser (unknown) #0-1] Spec: XXXXXXXXXX.spec.ts
[MultiremoteBrowser (unknown) #0-1] Running: MultiremoteBrowser on (unknown)
[MultiremoteBrowser (unknown) #0-1]
[MultiremoteBrowser (unknown) #0-1] XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-1]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-1]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-1]
[MultiremoteBrowser (unknown) #0-1] 2 passing (2.2s)
[MultiremoteBrowser (unknown) #0-1]
[MultiremoteBrowser (unknown) #0-1] 
[MultiremoteBrowser (unknown) #0-1] 
------------------------------------------------------------------
[MultiremoteBrowser (unknown) #0-2] Spec: XXXXXXXXXX.spec.ts
[MultiremoteBrowser (unknown) #0-2] Running: MultiremoteBrowser on (unknown)
[MultiremoteBrowser (unknown) #0-2]
[MultiremoteBrowser (unknown) #0-2] XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-2]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-2]
[MultiremoteBrowser (unknown) #0-2] 1 passing (4.2s)
[MultiremoteBrowser (unknown) #0-2]
[MultiremoteBrowser (unknown) #0-2] 
[MultiremoteBrowser (unknown) #0-2] 
------------------------------------------------------------------
[MultiremoteBrowser (unknown) #0-3] Spec: XXXXXXXXXX.spec.ts
[MultiremoteBrowser (unknown) #0-3] Running: MultiremoteBrowser on (unknown)
[MultiremoteBrowser (unknown) #0-3]
[MultiremoteBrowser (unknown) #0-3] XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-3]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-3]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-3]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-3]
[MultiremoteBrowser (unknown) #0-3] 3 passing (29s)
[MultiremoteBrowser (unknown) #0-3]
[MultiremoteBrowser (unknown) #0-3] 
[MultiremoteBrowser (unknown) #0-3] 
------------------------------------------------------------------
[MultiremoteBrowser (unknown) #0-4] Spec: XXXXXXXXXX.spec.ts
[MultiremoteBrowser (unknown) #0-4] Running: MultiremoteBrowser on (unknown)
[MultiremoteBrowser (unknown) #0-4]
[MultiremoteBrowser (unknown) #0-4] XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-4]    ✓ should XXXXXXXXXX
[MultiremoteBrowser (unknown) #0-4]
[MultiremoteBrowser (unknown) #0-4] 1 passing (4.8s)
[MultiremoteBrowser (unknown) #0-4]
[MultiremoteBrowser (unknown) #0-4] 
[MultiremoteBrowser (unknown) #0-4] 


Spec Files:	 5 passed, 1 total (500% completed) in 00:01:02 

Additional context
Excerpt from package.json:

  "scripts": {
    "test": "wdio"
  },
  "devDependencies": {
    "@types/mocha": "^7.0.2",
    "@types/node": "^12.12.47",
    "@wdio/cli": "^6.1.22",
    "@wdio/local-runner": "^6.1.22",
    "@wdio/mocha-framework": "^6.1.19",
    "@wdio/spec-reporter": "^6.1.14",
    "@wdio/sync": "^6.1.14",
    "chai": "^4.2.0",
    "chromedriver": "^83.0.0",
    "ts-node": "^8.10.2",
    "tsconfig-paths": "^3.9.0",
    "typescript": "^3.9.5",
    "wdio-chromedriver-service": "^6.0.3"
  }

[christian-bromann]

Hey,
thanks for filing the issue. This seems indeed like bug. We would appreciate any help on this so please help us out! Have a look at our contributing guidelines and reach out if you have any questions. Cheers!

[cjatkinson]

Investigating this issue.

[seanpoulter]

Any luck? Want a second set of eyes @cjatkinson? I've seen this too.

[christian-bromann]

@cjatkinson anything I can do to help?

[suniljaiswal01]

I would like to give a try on this. Is there any reproducible example available?

[suniljaiswal01]

@devopsix @christian-bromann The issue is happening only in case of multiRemote instance.
Following is my investigation:
When using multiRemote, the capabilities are not passed in array. Like below

capabilities: {

    myChromeBrowser: {
        capabilities: {
            browserName: 'chrome'
        }
    }

}

because of which the totalWorkerCnt value is set to "1" in the below code:
https://github.com/webdriverio/webdriverio/blob/master/packages/wdio-cli/src/launcher.js#L35-L39

TotalWorkerCnt is used to calculate to totalJobs, which is then used to calculate the "percentCompleted" value in the below code:
https://github.com/webdriverio/webdriverio/blob/master/packages/wdio-cli/src/interface.js#L244-L255

@christian-bromann could you please guide me on how this can be resolved

[cjatkinson]

Apologies, I ended up in the same place as you @suniljaiswal01. If I remember correctly @christian-bromann mentioned the multi-remote feature is being refactored?

[suniljaiswal01]

I think this PR #5458 will fix the issue . @christian-bromann could you confirm on this

[christian-bromann]

Ok, reviewing this issue it seems that the test uses Multiremote. This means all test have been running sequentially, not in parallel. This is why the spec reporter prints out total: 1 because there was only 1 worker active. I will close this but happy to discuss options how we can change wording to avoid such confusion.