tap runner is not working in node16 #746

ruyadorno · 2021-04-20T17:00:19Z

Tests are currently failing in consumers, such as the npm cli when running using node16:

npm t


> npm@7.10.0 test
> tap

Error: Cannot find module './reports/base'
Require stack:
- <path>/main/node_modules/jackspeak/noop.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:940:15)
    at resolveFileName (<path>/main/node_modules/tap/node_modules/resolve-from/index.js:17:39)
    at resolveFrom (<path>/main/node_modules/tap/node_modules/resolve-from/index.js:31:9)
    at module.exports (<path>/main/node_modules/tap/node_modules/resolve-from/index.js:34:41)
    at importJSX (<path>/main/node_modules/tap/node_modules/import-jsx/index.js:20:21)
    at module.exports (<path>/main/node_modules/tap/node_modules/treport/lib/index.js:13:18)
    at exports.makeReporter (<path>/main/node_modules/tap/bin/run.js:446:23)
    at runTests (<path>/main/node_modules/tap/bin/run.js:772:3)
    at mainAsync (<path>/main/node_modules/tap/bin/run.js:268:5)
    at main (<path>/main/node_modules/tap/bin/run.js:141:3) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '<path>/main/node_modules/jackspeak/noop.js'
  ]
}
TAP version 13
1..0
# time=1.271ms
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |
----------|---------|----------|---------|---------|-------------------

We can see tests are also currently failing in main branch of node-tap too:

$ npm t

> tap@15.0.3 test
> node bin/run.js -M coverage-map.js

TypeError: Cannot read property 'getFileName' of undefined
    at module.exports (/<path>/node-tap/node_modules/caller-path/index.js:4:40)
    at importJSX (/<path>/node-tap/node_modules/import-jsx/index.js:20:46)
    at module.exports (/<path>/node-tap/node_modules/treport/lib/index.js:13:18)
    at exports.makeReporter (/<path>/node-tap/bin/run.js:446:23)
    at runTests (/<path>/node-tap/bin/run.js:772:3)
    at mainAsync (/<path>/node-tap/bin/run.js:268:5)
TAP version 13
1..0
# time=1.428ms
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |
----------|---------|----------|---------|---------|-------------------

You can also see error on loading reporters when running node test/run/reporters.js

$ node test/run/reporters.js
TAP version 13
# Subtest: builtin reporter
    not ok 1 - Command failed: <path>/nodejs/out/Release/node <path>/node-tap/bin/run.js --no-coverage cli-tests-24496/ok.js -Rbase Error: Cannot find module './reports/base' Require stack: - <path>/node-tap/node_modules/jackspeak/noop.js     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:940:15)     at resolveFileName (<path>/node-tap/node_modules/resolve-from/index.js:17:39)     at resolveFrom (<path>/node-tap/node_modules/resolve-from/index.js:31:9)     at module.exports (<path>/node-tap/node_modules/resolve-from/index.js:34:41)     at importJSX (<path>/node-tap/node_modules/import-jsx/index.js:20:21)     at module.exports (<path>/node-tap/node_modules/treport/lib/index.js:13:18)     at exports.makeReporter (<path>/node-tap/bin/run.js:446:23)     at runTests (<path>/node-tap/bin/run.js:772:3)     at mainAsync (<path>/node-tap/bin/run.js:268:5)     at main (<path>/node-tap/bin/run.js:141:3) {   code: 'MODULE_NOT_FOUND',   requireStack: [     '<path>/node-tap/node_modules/jackspeak/noop.js'   ] }
      ---
      origin:
        at:
          line: 326
          column: 12
          file: node:child_process
          function: ChildProcess.exithandler
        killed: false
        code: 1
        signal: null
        cmd: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rbase
      found:
        !error
        name: Error
        message: >
          Command failed: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rbase

          Error: Cannot find module './reports/base'

          Require stack:

          - <path>/node-tap/node_modules/jackspeak/noop.js
              at Function.Module._resolveFilename (node:internal/modules/cjs/loader:940:15)
              at resolveFileName (<path>/node-tap/node_modules/resolve-from/index.js:17:39)
              at resolveFrom (<path>/node-tap/node_modules/resolve-from/index.js:31:9)
              at module.exports (<path>/node-tap/node_modules/resolve-from/index.js:34:41)
              at importJSX (<path>/node-tap/node_modules/import-jsx/index.js:20:21)
              at module.exports (<path>/node-tap/node_modules/treport/lib/index.js:13:18)
              at exports.makeReporter (<path>/node-tap/bin/run.js:446:23)
              at runTests (<path>/node-tap/bin/run.js:772:3)
              at mainAsync (<path>/node-tap/bin/run.js:268:5)
              at main (<path>/node-tap/bin/run.js:141:3) {
            code: 'MODULE_NOT_FOUND',
            requireStack: [
              '<path>/node-tap/node_modules/jackspeak/noop.js'
            ]
          }
        stack: |-
          Error:
              at ChildProcess.exithandler (node:child_process:326:12)
              at ChildProcess.emit (node:events:365:28)
              at maybeClose (node:internal/child_process:1067:16)
              at Socket.<anonymous> (node:internal/child_process:453:11)
              at Socket.emit (node:events:365:28)
              at Pipe.<anonymous> (node:net:661:12)
        killed: false
        code: 1
        signal: null
        cmd: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rbase
      at:
        line: 22
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:22:5
      source: |
        const check = t => (er, o, e) => {
          t.error(er)
        ----^
          o = clean(o)
            .replace(/^[\S\s]*SUMMARY RESULTS[\S\s]*$/,'treport output')
      ...

    not ok 2 - stdout
      ---
      compare: ===
      at:
        line: 26
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:26:5
      source: |2
            .replace(/^[\S\s]*✓[\S\s]*$/, 'spec output')
          t.matchSnapshot(o, 'stdout')
        ----^
          t.equal(clean(e), '', 'stderr')
          t.end()
      diff: |
        --- expected
        +++ actual
        @@ -1,1 +1,4 @@
        -treport output
        +TAP version 13
        +1..0
        +# {time}
        +
      ...

    not ok 3 - stderr
      ---
      found: |

        Error: Cannot find module './reports/base'
            {STACK}
        Require stack:
        - {CWD}/node_modules/jackspeak/noop.js
            at Function.Module._resolveFilename (node:internal/modules/cjs/loader#:#)
            at resolveFileName ({CWD}/node_modules/resolve-from/index.js#:#)
            at resolveFrom ({CWD}/node_modules/resolve-from/index.js#:#)
            at module.exports ({CWD}/node_modules/resolve-from/index.js#:#)
            at importJSX ({CWD}/node_modules/import-jsx/index.js#:#)
            at module.exports ({CWD}/node_modules/treport/lib/index.js#:#)
            at exports.makeReporter ({CWD}/bin/run.js#:#)
            at runTests ({CWD}/bin/run.js#:#)
            at mainAsync ({CWD}/bin/run.js#:#)
            at main ({CWD}/bin/run.js:141:3) {
          code: 'MODULE_NOT_FOUND',
          requireStack: [
            '{CWD}/node_modules/jackspeak/noop.js'
          ]
        }
      wanted: ""
      compare: ===
      at:
        line: 27
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:27:5
      source: |2
          t.matchSnapshot(o, 'stdout')
          t.equal(clean(e), '', 'stderr')
        ----^
          t.end()
        }
      ...

    1..3
    # failed 3 of 3 tests
not ok 1 - builtin reporter # time=390.392ms

# Subtest: tmr builtin reporter
    ok 1 - should not error
    ok 2 - stdout
    ok 3 - stderr
    1..3
ok 2 - tmr builtin reporter # time=274.866ms

# Subtest: cli reporter
    not ok 1 - Command failed: <path>/nodejs/out/Release/node <path>/node-tap/bin/run.js --no-coverage cli-tests-24496/ok.js -Rtap-parser -r-t -r-f TypeError [ERR_INVALID_ARG_TYPE]: The "listener" argument must be of type function. Received type string ('-f')     at checkListener (node:events:121:3)     at _addListener (node:events:398:3)     at Parser.addListener (node:events:456:10)     at Parser.on (<path>/node-tap/node_modules/minipass/index.js:335:20)     at new Parser (<path>/node-tap/node_modules/tap-parser/index.js:220:12)     at exports.makeReporter (<path>/node-tap/bin/run.js:469:18)     at runTests (<path>/node-tap/bin/run.js:772:3)     at mainAsync (<path>/node-tap/bin/run.js:268:5)     at main (<path>/node-tap/bin/run.js:141:3)     at Object.j.main (<path>/node-tap/node_modules/jackspeak/index.js:483:5) {   code: 'ERR_INVALID_ARG_TYPE' }
      ---
      origin:
        at:
          line: 326
          column: 12
          file: node:child_process
          function: ChildProcess.exithandler
        killed: false
        code: 1
        signal: null
        cmd: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rtap-parser -r-t -r-f
      found:
        !error
        name: Error
        message: >
          Command failed: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rtap-parser -r-t -r-f

          TypeError [ERR_INVALID_ARG_TYPE]: The "listener" argument must be of type function. Received type string ('-f')
              at checkListener (node:events:121:3)
              at _addListener (node:events:398:3)
              at Parser.addListener (node:events:456:10)
              at Parser.on (<path>/node-tap/node_modules/minipass/index.js:335:20)
              at new Parser (<path>/node-tap/node_modules/tap-parser/index.js:220:12)
              at exports.makeReporter (<path>/node-tap/bin/run.js:469:18)
              at runTests (<path>/node-tap/bin/run.js:772:3)
              at mainAsync (<path>/node-tap/bin/run.js:268:5)
              at main (<path>/node-tap/bin/run.js:141:3)
              at Object.j.main (<path>/node-tap/node_modules/jackspeak/index.js:483:5) {
            code: 'ERR_INVALID_ARG_TYPE'
          }
        stack: >-
          Error:
              at ChildProcess.exithandler (node:child_process:326:12)
              at ChildProcess.emit (node:events:365:28)
              at maybeClose (node:internal/child_process:1067:16)
              at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
        killed: false
        code: 1
        signal: null
        cmd: <path>/nodejs/out/Release/node
          <path>/node-tap/bin/run.js --no-coverage
          cli-tests-24496/ok.js -Rtap-parser -r-t -r-f
      at:
        line: 22
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:22:5
      source: |
        const check = t => (er, o, e) => {
          t.error(er)
        ----^
          o = clean(o)
            .replace(/^[\S\s]*SUMMARY RESULTS[\S\s]*$/,'treport output')
      ...

    not ok 2 - stdout
      ---
      compare: ===
      at:
        line: 26
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:26:5
      source: |2
            .replace(/^[\S\s]*✓[\S\s]*$/, 'spec output')
          t.matchSnapshot(o, 'stdout')
        ----^
          t.equal(clean(e), '', 'stderr')
          t.end()
      diff: |
        --- expected
        +++ actual
        @@ -1,6 +1,4 @@
         TAP version 13
        -ok 1 - cli-tests/ok.js > this is fine
        +1..0
         # {time}
        -1..1

        -
      ...

    not ok 3 - stderr
      ---
      found: >
        TypeError [ERR_INVALID_ARG_TYPE]: The "listener" argument must be of type
        function. Received type string ('-f')
            at checkListener (node:events#:#)
            at _addListener (node:events#:#)
            at Parser.addListener (node:events#:#)
            at Parser.on ({CWD}/node_modules/minipass/index.js#:#)
            at new Parser ({CWD}/node_modules/tap-parser/index.js#:#)
            at exports.makeReporter ({CWD}/bin/run.js#:#)
            at runTests ({CWD}/bin/run.js#:#)
            at mainAsync ({CWD}/bin/run.js#:#)
            at main ({CWD}/bin/run.js#:#)
            at Object.j.main ({CWD}/node_modules/jackspeak/index.js:483:5) {
          code: 'ERR_INVALID_ARG_TYPE'
        }
      wanted: ""
      compare: ===
      at:
        line: 27
        column: 5
        file: test/run/reporters.js
      stack: |
        test/run/reporters.js:27:5
      source: |2
          t.matchSnapshot(o, 'stdout')
          t.equal(clean(e), '', 'stderr')
        ----^
          t.end()
        }
      ...

    1..3
    # failed 3 of 3 tests
not ok 3 - cli reporter # time=179.496ms

# Subtest: stream reporter
    ok 1 - should not error
    ok 2 - stdout
    ok 3 - stderr
    1..3
ok 4 - stream reporter # time=282.474ms

# Subtest: react component
    ok 1 - should not error
    ok 2 - stdout
    ok 3 - stderr
    1..3
ok 5 - react component # time=560.124ms

# Subtest: failures
    ok 1 - expected to throw
    ok 2 - expected to throw
    ok 3 - expected to throw
    1..3
ok 6 - failures # time=79.53ms

1..6
# failed 2 of 6 tests
# time=1775.193ms

More info

node v16.0.0
npm v7.10.0

The text was updated successfully, but these errors were encountered:

ruyadorno · 2021-04-20T18:02:12Z

Here's a related user report in an issue comment: #624 (comment)

ruyadorno · 2021-04-20T18:04:03Z

Also related, thread tracking this problem in Node.js smoke tests' repo (citgm): nodejs/citgm#852 (comment)

ruyadorno · 2021-04-20T18:41:36Z

Should be fixed by a73b158

trentm · 2021-10-28T17:58:40Z

This is still broken in tap@14 because it is using treport@1. treport doesn't have a changelog or an "engines" in its package.json so I can't easily tell what breaking change there was in treport@2.

Could tap@14 be updated to treport@2? Is there any intention in maintaining tap@14 anymore? If not, then currently it is impossible to use node-tap with a project that wants to test with node v8 and node v16. Granted it is time to drop node v8 support, but wheels grind slowly sometime. :)

Thanks.

isaacs · 2021-10-28T18:08:14Z

@trentm The problem is that treport 2 uses libtap under the hood, so it can never work with node 8.

Really, the core problem is some incompatibilities in some lowlevel stuff that changed between node 8 and node 16, making it very difficult to bridge the gap. I can't fault them for having some breaking changes across 8 SemVer major bumps, of course, but it does make this unresolvable.

I recommend using tap 15, but have a pre-test command in your CI workflows to manually install tap 14 on node 8. It'll mean that you have to be a little careful about which features you use in your tests, but I don't think very much changed in that part of the interface (and was mostly additive), so it should be relatively easy. The breaking changes were all in the test harness/runner machinery and adding strictness to a bunch of things, but tests themselves should be mostly backwards compatible.

trentm · 2021-10-28T18:13:34Z

@isaacs Thanks. Yah... I got lost in tracing the differences in CallSite.getTypeName() (from https://v8.dev/docs/stack-trace-api) in the caller-path and caller-callsite packages used by import-jsx between node v14 and v16. I could get things to reliably break in node v14 as well by adding or remove 'use strict' at the top of some of:

caller-callsite/index.js
caller-path/index.js
import-jsx/index.js
treport/index.js

The v8.dev link above states "To maintain restrictions imposed on strict mode functions, frames that have a strict mode function and all frames below (its caller etc.) are not allow to access their receiver and function objects. "
I suspect that is related, but I stopped digging.

Anyway, I understand that it is unresolvable in tap 14. Cheers.

Also bump to tap@15 (min supported node is v10). tap@14 doesn't work with node v16 and later (tapjs/tapjs#746).

ruyadorno closed this as completed Apr 20, 2021

clemyan mentioned this issue May 3, 2021

Broken V8 source map cache file URL parsing on Windows istanbuljs/v8-to-istanbul#143

Closed

BethGriggs mentioned this issue May 7, 2021

chore: test using node 16, remove node 10 nodejs/citgm#862

Merged

4 tasks

kemayo mentioned this issue Jun 29, 2021

Update to use emojilib v3 jsumners/alfred-emoji#60

Closed

sify21 mentioned this issue Sep 18, 2021

update tap to latest to run test locally npm/node-semver#401

Closed

trentm added a commit to elastic/ecs-logging-nodejs that referenced this issue Oct 28, 2021

tests: Test with node v17

d930978

Also bump to tap@15 (min supported node is v10). tap@14 doesn't work with node v16 and later (tapjs/tapjs#746).

trentm mentioned this issue Oct 28, 2021

tests: Test with node v17 elastic/ecs-logging-nodejs#107

Merged

trentm added a commit to elastic/ecs-logging-nodejs that referenced this issue Oct 28, 2021

tests: Test with node v17 (#107)

222d6b3

Also bump to tap@15 (min supported node is v10). tap@14 doesn't work with node v16 and later (tapjs/tapjs#746).

imcotton mentioned this issue Jan 30, 2022

Refine dependencies purescript/npm-installer#33

Merged

F3n67u mentioned this issue Jul 23, 2022

Update dependency tap to v16 nodejs/node-pr-labeler#15

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tap runner is not working in node16 #746

tap runner is not working in node16 #746

ruyadorno commented Apr 20, 2021

ruyadorno commented Apr 20, 2021

ruyadorno commented Apr 20, 2021

ruyadorno commented Apr 20, 2021

trentm commented Oct 28, 2021

isaacs commented Oct 28, 2021

trentm commented Oct 28, 2021

tap runner is not working in node16 #746

tap runner is not working in node16 #746

Comments

ruyadorno commented Apr 20, 2021

More info

ruyadorno commented Apr 20, 2021

ruyadorno commented Apr 20, 2021

ruyadorno commented Apr 20, 2021

trentm commented Oct 28, 2021

isaacs commented Oct 28, 2021

trentm commented Oct 28, 2021