Confusing interaction between --parallel/--split and testem parallel option

[dwickern]

I am confused about how --parallel interacts with the parallel option in testem.js. I have my tests running in 3 browsers (chrome, firefox, ie). Testem's parallel option works how I expect:

ember test with parallel: 1 (default) runs tests sequentially
ember test with parallel: 2 runs tests in e.g. chrome and firefox first, then iexplore afterwards
ember test with parallel: -1 runs all 3 browsers at the same time

But using ember-exam I get some unexpected behavior:

ember exam --parallel --split=3 with parallel: 1 runs tests sequentially like ember test 🤔
ember exam --parallel --split=3 with parallel: 3 uses 3 browser instances as expected
ember exam --parallel --split=3 with parallel: -1 uses 9 browser instances 🤔

Is there some circumstance where this behavior makes sense?
Maybe ember exam --split should also set the testem --parallel option?

[trentmwillis]

When using --parallel with --split, Ember Exam configures Testem to use multiple test pages, which so that the tests are split up across different processes. The way Testem works is that for each test page, you'll get one instance of each launcher you've defined. So, the behavior you're seeing is default from Testem. You can read more here.

If you feel this needs better clarification, please open a PR to the readme. Thanks!

[dwickern]

What do you think about adding testem's parallel option to the CLI? It could be an argument to the existing --parallel option:

exam --parallel=3 --split=3 would split and run up to 3 browser instances
exam --parallel=-1 --split=3 would run unlimited browser instances
exam --parallel=3 would run 3 browser instances without splitting the tests