Asynchronous tests cause "test exited without ending"

[jaspervdg]

The following minimal example produces a "test exited without ending" error on my system (tape 4.0.0 on node 0.12.2):

var test = require("tape")

test("synch", function(t) { t.end() })

setTimeout(function() {
    test("synch", function(t) { t.end() })
}, 100)

Interestingly, this same problem also turns up when using tape on the command line to run multiple test scripts, one of which runs its test asynchronously, even though when run individually all test scripts work just fine.

[Raynos]

Do not asynchronously declare tests. All tests should be declared synchronously in the same tick.

[jaspervdg]

I'm afraid that's not really an option. The actual set up is that we browserify some tests and then run them. And since browserify asynchronously loads files, etc., I don't really see how we could run those tests synchronously. I did manage to at least run all the browserify tests "synchronously" with respect to each other (by waiting for all browserify calls to finish, and then running them), but this approach breaks down when running a set of tests (which includes the browserify tests) using the tape commandline tool, since the tests still run within an (asynchronous) callback function (note that running them with tap does work, probably because of the different approach to invoking the individual test scripts).

[ghost]

If you need to perform setup and teardown, you can do this inside your tests.

Here's a nice way to do that: #59 (comment)

You should start your tests on the first tick, but they don't need to finish on the first tick, they can take as long as they need until ending with t.end() or reaching their planned number of assertions.

[jaspervdg]

That indeed works (although here I had to nest tests, given the set up we have). It might be nice to document this particular aspect of tape somewhere.

[jbcpollak]

Is there any way to asynchronously declare tests? I'd like to use a streaming (reactive) library to scan my directories for tests and instantiate them. I can image this would be useful for watching a directory for changes as well.

[nfcampos]

not sure this is what you mean but i do the following:

import tape from 'tape'

class T {
  constructor() { this.tests = [] }
  add(...args) { this.tests.push(args) }
  run() { this.tests.map(args => tape(...args)) }
}

const tests = new T()

tests.add('some test', t => {
  t.ok(true)
  t.end()
})

tests.run()

this way calls to tests.add() can be asynchronous because tests are still declared to tape all on the same tick (only when you call tests.run())

[jbcpollak]

Interesting, so you need to require you T class in your tests to run tests.add() right?

I wanted to load tests at the file level, so what I do is something like this (I'm using RxJS, so this is just pseudo-code, its too verbose otherwise):

var testDirs = [...];
source = from(testDirs)
   .flatMap(testDir -> toGlob)
   .filter( /* remove globs with no tests */);

source.subscribe() {
   function(testFile) {
       // onNext
       tests.push(testFile);
   },
   onErrorFn,
   function() {
      // onComplete
      for(testFile in tests) {
         require(testFile);
      }
   });

This is probably more trouble than its worth, but it lets me setup output filters in onComplete to convert to xUnit format and then write to file.

I guess it has the same effective result as your solution.

I just wish you could stream each test without having to build up the array of test files (in my case) or test objects (in yours).

[nfcampos]

i actually do something like this:

//t.js
import tape from 'tape'

export default class T {
  constructor() { this.tests = [] }
  add(...args) { this.tests.push(args) }
  run() { this.tests.map(args => tape(...args)) }
}

//some-test.js
export const someTest = ['some test', t => {
  t.ok(true)
  t.end()
}]

//tests.js
import T from './t'

export const tests = new T()

import { someTest, someOtherTest, yetAnotherTest } from './some-test'
tests.add(...someTest)
tests.add(...someOtherTest)
tests.add(...yetAnotherTest)

tests.run()

so all test files are a bunch of exports and tests.jsends up being an index of all my tests...

[zetekla]

@nfcampos does that work for you?

[nfcampos]

Yes it does.

[ljharb]

Sounds like this issue can be closed.

tl;dr: all test cases must be synchronously declared.