Run multiple intern.run() with API

[AmirTugi]

The Problem

When trying to run intern more than one or even the same intern more than once, using intern.run(), after re-configuring it, it will shout that it cannot run more than once + due to intern using global scope - cannot run more than 1 in parallel.

This behaviour is restricting the user from running and handling intern as he wishes (in our case, make a grid of intern runners who run parallelly)

Possible solution

When removing the restriction, and maybe make a cleanup (also using other, more isolated scopes for resources) of resources after the intern.run() we could run multiple interns parallelly, and we'll have more freedom and flexability with intern's api

Steps to reproduce (for bugs)

Just try to run the same suites twice, either parallelly, or in series.

Environment

Intern version: 4.2.0

[jason0x43]

The current plan for running Intern in parallel is to use multiple node processes. Intern would start up a separate node process to run tests, much like it starts a browser to run tests now. This will allow for the same logic to manage tests in Node and browsers, and will provide isolation between parallel Node sessions.

[AlonMiz]

A related issue is that you cannot run parallel functional tests at all.
Unless they are configured on different browsers - which means I can't run 2 tests in parallel on chrome v66 for example.

[jason0x43]

Yeah, we have #546 for sharding across multiple instances of a single environment and #803 for running Node as a standard environment. Between the two of those we should end up with a pretty nice parallelization platform.

[AlonMiz]

@jason0x43 any news? :/
currently, our solution is to spawn multiple child node processes, get the stdout and stderr, and parse the output to figure out fail and success suites and tests.
this is really hard and prone to errors, and really ugly.

[jason0x43]

It's high in the queue, but I haven't had time to get to it yet.

In the meantime you might consider writing a custom reporter that will output test results in a structured format, like JSON. That will make result aggregation simpler.

[jacobroufa]

Bumping this one up as I've recently run into a similar issue. I am using the same technique as @AlonMiz to usable effect, though it is kludgy as suggested.