[mocha?] client.perform() does not handle most exceptions correctly (or at all in some cases)

[mjbogusz]

Observed behaviour:

client.perform() does not handle most exceptions thrown inside of it, either by swallowing them up (thus passing the test, aka false-positive) or by letting them outside its scope (thus crashing whole process and possibly leaving behind a never-terminated selenium session/process).

Expected behaviour:

client.perform() should handle all exceptions (and possibly rejections) happening inside and fail proper test case, just like mocha's it() handles it.

Test scenario:

All files are in this gist: https://gist.github.com/mjbogusz/47d92607551b556d1d5a78613194463e

• test file (place it in src directory): https://gist.github.com/mjbogusz/47d92607551b556d1d5a78613194463e#file-nightwatch_perform_test-js
• Nightwatch config (start selenium process, use firefox, use mocha): https://gist.github.com/mjbogusz/47d92607551b556d1d5a78613194463e#file-nightwatch-json
• output: https://gist.github.com/mjbogusz/47d92607551b556d1d5a78613194463e#file-output-log
• verbose output: https://gist.github.com/mjbogusz/47d92607551b556d1d5a78613194463e#file-output_verbose-log

Environment

Tested in a clear environment: fresh nvm, no global modules, only npm install nightwatch selenium-standalone mocha and then node_modules/.bin/selenium_standalone install --drivers.firefox.version=0.18.0. Verified to happen on chrome-driver too.
Versions (all up-to-date):

• nodejs: 8.2.1
• nightwatch: 0.9.16
• mocha: 3.4.2
• selenium: 3.4.0 (geckodriver: 0.18.0, chromedriver: 2.31)
• OS: tested on Debian Buster (testing) and Archlinux, both up-to-date.

Additional info

I'm not sure if this behaviour is anyhow connected to mocha, I'll try to re-write the test file for 'pure' Nightwatch soon.

The 'skipped' tests are the ones that crash the whole process.

I want to additionally point out two intriguing behaviours:

• cases 1 & 3: assert() throws an error, but somehow it's handled differently from just throwing one manually
• cases 8 & 9: 'asynchronous' error from Nightwatch's client.assert.ok() is handled, while one from regular assert() is not; but when happening 'synchronously' (2 & 3) they're both handled properly.

[vcshox]

Meet the same problem ...

[stale]

This issue has been automatically marked as stale because it has not had any recent activity.
If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.