Support to randomise tests

[pke]

Catching issues caused by globals are easier to catch if node-tap would allow to run tests within a testfile in random order. I know that it does that when you put the tests in different files you can run them in parallel which would also surface such global vars bugs. However it would be nice to be able to randomise tests within a file too.

[isaacs]

In general, I very much prefer to avoid situations where tests pass "sometimes", but not other times. Coverage, code structure, and careful thinking can help ensure that the program is being well covered, and fuzz testing like this is often frustrating, making it easy to justify ignoring a failing test. (Run it again, it'll be fine. Just a fluke. Etc.)

You can of course do this yourself using parallel tests the same as you can with test files. Just set t.jobs = <some number> and then any async tests will be run in parallel. See for yourself:

const t = require('./')
t.jobs = 10

// run for some random amount of time up to 1s
const asyncRandomTest = (t) => {
  console.error('starting ', t.name)
  setTimeout(() => {
    console.error('ending', t.name)
    t.end()
  }, 1000 * Math.random())
}

t.test('first', asyncRandomTest)
t.test('second', asyncRandomTest)
t.test('third', asyncRandomTest)
t.test('fourth', asyncRandomTest)