Mocha test failures missing inline error reporting / output inspector / stack trace

[bompus]

I noticed that when I was testing Mocha, that I was not seeing any inline error reporting for failed tests. I saw these inline errors and the output inspector had content when using Jest and AVA.

I also noticed that when looking at Output -> Wallaby.js Tests, there was no stack trace with Mocha, but Jest and AVA had one, so maybe it's a parsing issue?

Expected: https://wallabyjs.com/docs/intro/get-started-vscode.html#error-reporting ( error line display, hover, output inspector - all blank or missing )

Reproduction repo: https://github.com/bompus/wallaby-mocha-esm

The reproduction repo tests failures using assertions from node assert, chai's expect, and jest's expect. All exhibit the same described issue.

Wallaby diagnostics report

{
  editorVersion: '1.60.2',
  pluginVersion: '1.0.313',
  editorType: 'VSCode',
  osVersion: 'linux 5.11.0-34-generic',
  nodeVersion: 'v16.10.0',
  coreVersion: '1.0.1149',
  checksum: 'ZjhjMmFjNWM1MGNjOGMzNzk3NWNhYWMxOWQ3MTJjNDcsMTY2MTgxNzYwMDAwMCww',
  config: {
    testFramework: { version: 'mocha@2.1.0', configurator: 'mocha@2.1.0', reporter: 'mocha@2.1.0', starter: 'mocha@2.1.0' },
    files: [
      { pattern: 'package.json', ignore: false, trigger: true, load: true, instrument: true, order: 1 },
      { pattern: 'src/**/*.js', ignore: false, trigger: true, load: true, instrument: true, order: 2 },
      { pattern: 'src/**/*.test.js', ignore: true, trigger: true, load: true, instrument: true }
    ],
    tests: [ { pattern: 'src/**/*.test.js', ignore: false, trigger: true, load: true, test: true, order: 3 } ],
    env: { type: 'node', runner: 'node', params: { runner: '--experimental-vm-modules' }, viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    diagnostics: {},
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    addModifiedTestFileToExclusiveTestRun: true,
    compilers: { '**/*.?(lit)coffee?(.md)': [Function (anonymous)] },
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: true,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: undefined,
    symlinkNodeModules: true,
    testLocations: [ 'src/a.test.js' ],
    configCode: '/***\n' +
      ' ** In VS Code, must execute Ctrl+Shift+P -> Wallaby.js: Select Configuration -> wallaby.js for it to start using this file **\n' +
      ' ***/\n' +
      '\n' +
      'export default function () {\n' +
      '  return {\n' +
      "    testFramework: 'mocha',\n" +
      "    files: ['package.json', 'src/**/*.js', '!src/**/*.test.js'],\n" +
      "    tests: ['src/**/*.test.js'],\n" +
      '    env: {\n' +
      "      type: 'node',\n" +
      "      runner: 'node',\n" +
      '      params: {\n' +
      "        runner: '--experimental-vm-modules',\n" +
      '      },\n' +
      '    },\n' +
      '  };\n' +
      '}\n'
  },
  packageJSON: { dependencies: undefined, devDependencies: { chai: '^4.3.4', expect: '^27.2.1', mocha: '9.1.1' } },
  fs: { numberOfFiles: 2 },
  debug: [
    '2021-09-23T16:20:25.142Z project Wallaby Node version: v16.10.0\n',
    '2021-09-23T16:20:25.142Z project Wallaby config: <rootDir>/wallaby.js\n',
    '2021-09-23T16:20:25.169Z project File cache: <homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/projects/42175b3cc768421d\n',
    '2021-09-23T16:20:25.219Z uiService Listening port 51235\n',
    '2021-09-23T16:20:25.228Z workers Parallelism for initial run: 4, for regular run: 2\n',
    '2021-09-23T16:20:25.228Z workers Starting run worker instance #0\n',
    '2021-09-23T16:20:25.228Z workers Starting run worker instance #1\n',
    '2021-09-23T16:20:25.228Z workers Starting run worker instance #2\n',
    '2021-09-23T16:20:25.228Z workers Starting run worker instance #3\n',
    '2021-09-23T16:20:25.229Z workers Web server is listening at 10989\n',
    '2021-09-23T16:20:25.253Z project Stopping process pool\n',
    '2021-09-23T16:20:25.253Z project File cache is up-to-date, starting full test run\n',
    '2021-09-23T16:20:25.254Z project Running postprocessor\n',
    '2021-09-23T16:20:25.256Z postprocessor New TypeScript language service is required\n',
    "2021-09-23T16:20:25.256Z postprocessor TypeScript postprocessor will be removed because in 2 project files none were found by patterns: [ '**/*.ts?(x)' ]\n",
    '2021-09-23T16:20:25.258Z project Postprocessor execution finished\n',
    '2021-09-23T16:20:25.258Z project Postprocessor is removed as requested by itself\n',
    '2021-09-23T16:20:25.258Z project Test run started; run priority: 3\n',
    '2021-09-23T16:20:25.258Z project Running all tests\n',
    '2021-09-23T16:20:25.259Z workers Starting test run, priority: 3\n',
    '2021-09-23T16:20:25.259Z nodeRunner Starting sandbox [worker #0, session #qb7r0]\n',
    '2021-09-23T16:20:25.259Z nodeRunner Preparing sandbox [worker #0, session #qb7r0]\n',
    '2021-09-23T16:20:25.334Z workers Started run worker instance (delayed) #0\n',
    '2021-09-23T16:20:25.335Z nodeRunner Prepared sandbox [worker #0, session #qb7r0]\n',
    '2021-09-23T16:20:25.335Z workers [worker #0, session #qb7r0] Running tests in sandbox\n',
    '2021-09-23T16:20:25.339Z workers Started run worker instance (delayed) #1\n',
    '2021-09-23T16:20:25.343Z workers Started run worker instance (delayed) #2\n',
    '2021-09-23T16:20:25.357Z workers Started run worker instance (delayed) #3\n',
    '2021-09-23T16:20:25.409Z workers [qb7r0] Loaded 6 test(s)\n',
    '2021-09-23T16:20:25.412Z workers [qb7r0] Test executed: assert - should pass\n',
    '2021-09-23T16:20:25.413Z workers [qb7r0] Test executed: assert - should fail\n',
    '2021-09-23T16:20:25.414Z workers Failed to map the stack to user code, entry message: Expected values to be strictly equal:\n' +
      '+ actual - expected\n' +
      '\n' +
      "+ 'string'\n" +
      "- 'number', stack: AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:\n" +
      '+ actual - expected\n' +
      '\n' +
      "+ 'string'\n" +
      "- 'number'\n" +
      '    at Context.<anonymous> (file://<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/projects/42175b3cc768421d/instrumented/src/a.test.js?update=1632414025356:15:34)\n' +
      '    at callFn (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:366:21)\n' +
      '    at Test.Runnable.run (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:354:5)\n' +
      '    at Runner.n.runTest (<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/wallaby95fded/runners/node/mocha@2.1.0/initializer.js:14:2548)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:804:12\n' +
      '    at next (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:596:14)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:606:7\n' +
      '    at next (/tmp/wallaby-mocha-es\n',
    '2021-09-23T16:20:25.415Z workers [qb7r0] Test executed: chai.expect - should pass\n',
    '2021-09-23T16:20:25.415Z workers [qb7r0] Test executed: chai.expect - should fail\n',
    "2021-09-23T16:20:25.416Z workers Failed to map the stack to user code, entry message: expected 'string' to equal 'number', stack: AssertionError: expected 'string' to equal 'number'\n" +
      '    at Context.<anonymous> (file://<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/projects/42175b3cc768421d/instrumented/src/a.test.js?update=1632414025356:23:52)\n' +
      '    at callFn (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:366:21)\n' +
      '    at Test.Runnable.run (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:354:5)\n' +
      '    at Runner.n.runTest (<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/wallaby95fded/runners/node/mocha@2.1.0/initializer.js:14:2548)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:804:12\n' +
      '    at next (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:596:14)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:606:7\n' +
      '    at next (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runne\n',
    '2021-09-23T16:20:25.418Z workers [qb7r0] Test executed: jest.expect - should pass\n',
    '2021-09-23T16:20:25.418Z workers [qb7r0] Test executed: jest.expect - should fail\n',
    '2021-09-23T16:20:25.418Z workers Failed to map the stack to user code, entry message: expect(received).toStrictEqual(expected) // deep equality\n' +
      '\n' +
      'Expected: "number"\n' +
      'Received: "string", stack: Error: expect(received).toStrictEqual(expected) // deep equality\n' +
      '\n' +
      'Expected: "number"\n' +
      'Received: "string"\n' +
      '    at Context.<anonymous> (file://<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/projects/42175b3cc768421d/instrumented/src/a.test.js?update=1632414025356:31:50)\n' +
      '    at callFn (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:366:21)\n' +
      '    at Test.Runnable.run (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runnable.js:354:5)\n' +
      '    at Runner.n.runTest (<homeDir>/.vscode-server/extensions/wallabyjs.wallaby-vscode-1.0.313/wallaby95fded/runners/node/mocha@2.1.0/initializer.js:14:2548)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:804:12\n' +
      '    at next (<rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:596:14)\n' +
      '    at <rootDir>/node_modules/.pnpm/mocha@9.1.1/node_modules/mocha/lib/runner.js:606:7\n' +
      '    at next (<rootDir>/node_mo\n',
    '2021-09-23T16:20:25.420Z workers [qb7r0] Run 6 test(s), skipped 0 test(s)\n',
    '2021-09-23T16:20:25.422Z workers [qb7r0] Sandbox is responsive, closing it\n',
    '2021-09-23T16:20:25.422Z project Test run finished\n',
    '2021-09-23T16:20:25.422Z project Processed console.log entries\n',
    '2021-09-23T16:20:25.422Z project Processed loading sequences\n',
    '2021-09-23T16:20:25.423Z project Processed executed tests\n',
    '2021-09-23T16:20:25.423Z project Processed code coverage\n',
    '2021-09-23T16:20:25.431Z project Test run result processed and sent to IDE\n'
  ]
}

[smcenlly]

Thanks for reporting the issue. Wallaby was not correctly mapping stack traces reported from mocha with esm when reported in the format:

"at Context. (file:///..."

This has been fixed in Wallaby core v1.0.1150. Please update to this version and it will fix your problem.

[bompus]

All better now. Thanks for the quick update!