before() this.skip() causes hangs, incomplete testing and error

[mwri]

Issue description or question

Bought Wallaby yesterday, I'm using vscode and after some pain, I discovered that calling this.skip() from before really messes up Wallaby. About half the time it fails to complete at all (the little animation at the bottom right wizzing away forever and the logs saying a test is taking a long time) and when it does complete it reports 0 failing tests, 157 passing but there are 210 tests!

In addition to the report, if it finishes, the following error is emitted:

TypeError: Cannot read property 'length' of undefined 
    at emitOne (events.js:121:20) 
    at Runner.emit (events.js:211:7) 
    at next (./node_modules/mocha/lib/runner.js:631:12) 
    at Runner.runTests (./node_modules/mocha/lib/runner.js:694:3) 
    at ./node_modules/mocha/lib/runner.js:783:10 
    at next (./node_modules/mocha/lib/runner.js:362:14) 
    at Immediate._onImmediate (./node_modules/mocha/lib/runner.js:425:5) 
    at runCallback (timers.js:789:20) 
    at tryOnImmediate (timers.js:751:5) 
    at processImmediate [as _immediateCallback] (timers.js:722:5) 

The cause is one of my test files contains:

describe('integration', function () {
    this.timeout(20000);

    before(async function () {
        if (process.env.MYAPP_TEST_INTEGRATION !== 'true')
            return this.skip();

        // SNIP setup for integration
    });

    // SNIP integration tests
});

The integration tests spin up various docker containers and are quite slow, hence the normal exclusion. Anyway, it turns out that the this.skip() is the cause of all the pain, I can comment out the entire file contents and it still does it as long as that's still there. I haven't experimented to see what other contexts skipping is problematic for.

Fortunately the integration tests are all in one file, so I've excluded it and all the problems went away.

You probably don't need more information, but, my config is:

process.env.MYAPP_CONFIG = './test/test.conf';

module.exports = function () {
    return {
        files: [
            'lib/**',
            'test/test.conf',
            'db/**',
            'webapp/**',
        ],
        tests: [
            'test/**/*.js'
        ],
        env: {
            type: 'node',
        },
        debug: true,
        trace: true,
    };
}

And, logging:

[Info]  2020-02-17T10:40:21.921Z project Wallaby Node version: v8.9.1
[Info]  2020-02-17T10:40:21.921Z project Wallaby config: ********************/.wallaby.conf.js
[Info]  Started Wallaby.js Core v1.0.845
[Info]  2020-02-17T10:40:21.971Z fs File added: **************
...
...
[Info]  2020-02-17T10:40:23.895Z workers [kzpkz] Test executed: ********************************
[Info]  Execution progress: 100 tests
[Info]  2020-02-17T10:40:23.898Z workers [kzpkz] Test executed: ********************************
[Info]  2020-02-17T10:40:23.898Z workers [kzyd8] Test executed: ********************************
[Info]  2020-02-17T10:40:23.898Z workers [kzpkz] Test executed: ********************************
[Info]  2020-02-17T10:40:23.900Z workers Sandbox (active) [kzpkz] error: Cannot read property 'length' of undefined
[Error] Runtime error: TypeError: Cannot read property 'length' of undefined 
[Error]     at emitOne (events.js:121:20) 
[Error]     at Runner.emit (events.js:211:7) 
[Error]     at next (./node_modules/mocha/lib/runner.js:631:12) 
[Error]     at Runner.runTests (./node_modules/mocha/lib/runner.js:694:3) 
[Error]     at ./node_modules/mocha/lib/runner.js:783:10 
[Error]     at next (./node_modules/mocha/lib/runner.js:362:14) 
[Error]     at Immediate._onImmediate (./node_modules/mocha/lib/runner.js:425:5) 
[Error]     at runCallback (timers.js:789:20) 
[Error]     at tryOnImmediate (timers.js:751:5) 
[Error]     at processImmediate [as _immediateCallback] (timers.js:722:5) 
[Info]  2020-02-17T10:40:23.902Z workers Failed to map the stack to user code, entry message: Cannot read property 'length' of undefined, stack: TypeError: Cannot read property 'length' of undefined
[Info]      at Runner.<anonymous> (/home/mjw/.vscode/extensions/wallabyjs.wallaby-vscode-1.0.174/wallaby/runners/node/mocha@2.1.0/initializer.js:14:2139)
[Info]      at emitOne (events.js:121:20)
[Info]      at Runner.emit (events.js:211:7)
[Info]      at next (APP_DIR/node_modules/mocha/lib/runner.js:631:12)
[Info]      at Runner.runTests (APP_DIR/node_modules/mocha/lib/runner.js:694:3)
[Info]      at APP_DIR/node_modules/mocha/lib/runner.js:783:10
[Info]      at next (APP_DIR/node_modules/mocha/lib/runner.js:362:14)
[Info]      at Immediate._onImmediate (APP_DIR/node_modules/mocha/lib/runner.js:425:5)
[Info]      at runCallback (timers.js:789:20)
[Info]      at tryOnImmediate (timers.js:751:5)
[Info]      at processImmediate [as _immediateCallback] (timers.js:722:5)
[Info]  2020-02-17T10:40:23.903Z workers [kzyd8] Test executed: ********************************
...
...
[Info]  Finished executing 157 affected test(s) in 1.47 second(s)
[Info]  2020-02-17T10:40:24.273Z project Test run finished
[Info]  2020-02-17T10:40:24.274Z project Processed console.log entries
[Info]  2020-02-17T10:40:24.274Z project Processed loading sequences
[Info]  2020-02-17T10:40:24.275Z project Test name duplicate: denys to user with incorrect password
[Info]  2020-02-17T10:40:24.275Z project Test name duplicate: grants write access to write access user
[Info]  2020-02-17T10:40:24.276Z project Processed executed tests
[Info]  2020-02-17T10:40:24.415Z project Processed code coverage
[Info]  2020-02-17T10:40:25.139Z project Test run result processed and sent to IDE

[ArtemGovorov]

Thanks for providing the details. The issue is fixed and the fix is published in the latest Wallaby Core v1.0.846. Your core version will be updated automatically, however to get the fix ASAP you may use Wallaby.js Force Core Update command in VS Code to get the fix immediately.

[mwri]

Are you sure @ArtemGovorov?

I tried removing the exception for my integration tests file today and it behaved as before:

[Info]  Started Wallaby.js Core v1.0.852
[Info]  Wallaby App (realtime reports) is available at: http://localhost:51245
[Info]  Execution progress: 50 tests
[Info]  Execution progress: 100 tests
[Info]  Execution progress: 150 tests
[Error] Runtime error: TypeError: Cannot read property 'length' of undefined 
[Error]     at emitOne (events.js:121:20) 
[Error]     at Runner.emit (events.js:211:7) 
[Error]     at next (./node_modules/mocha/lib/runner.js:631:12) 
[Error]     at Runner.runTests (./node_modules/mocha/lib/runner.js:694:3) 
[Error]     at ./node_modules/mocha/lib/runner.js:783:10 
[Error]     at next (./node_modules/mocha/lib/runner.js:362:14) 
[Error]     at Immediate._onImmediate (./node_modules/mocha/lib/runner.js:425:5) 
[Error]     at runCallback (timers.js:789:20) 
[Error]     at tryOnImmediate (timers.js:751:5) 
[Error]     at processImmediate [as _immediateCallback] (timers.js:722:5) 
[Info]  Finished executing 157 affected test(s) in 1.7 second(s)

Putting the exception back in again:

[Info]  Started Wallaby.js Core v1.0.852
[Info]  Wallaby App (realtime reports) is available at: http://localhost:51245
[Info]  Execution progress: 50 tests
[Info]  Execution progress: 100 tests
[Info]  Execution progress: 150 tests
[Info]  Execution progress: 200 tests
[Info]  Finished executing 210 affected test(s) in 2.04 second(s)

[smcenlly]

@mwri - please ignore this comment - I had misread your core version

[ArtemGovorov]

Thanks for checking. We have not considered one scenario that is likely to break in your case. The issue is also fixed and the fix is published in the latest Wallaby Core v1.0.853. Your core version will be updated automatically, however to get the fix ASAP you may use Wallaby.js Force Core Update command in VS Code to get the fix immediately.

[mwri]

Yep, confirmed, I can enable the integration tests without prejudice now, thanks.