Support specifying multiple partitions #63
Conversation
The primary use case is splitting test modules across multiple CI containers, and having each of those CI containers run its list of tests in multiple parallel runs (you would specify split, parallel, and multiple partitions for each container). This can also be used for unbalanced partitioning, e.g. if you want one container to run 2/3 of the tests and the other run 1/3 (you would specify split and then different numbers of partitions for different containers).
|
Thanks for the work on this! A quick glance looks pretty good. I'll review in depth some time this weekend, if you could look into the codeclimate issue before then it'd be appreciated. |
|
I think this is all ready to go now. |
| @@ -98,6 +102,8 @@ ok 2 PhantomJS 1.9 - Exam Partition #3 - some other other test | |||
| ok 3 PhantomJS 1.9 - Exam Partition #2 - some other test | |||
| ``` | |||
|
|
|||
| You can also combine the `parallel` option with the `partition` option to split tests, and then recombine partitions into parallel runs. This would, for example, allow you to run tests in multiple CI containers and have each CI container parallelize its list of tests. | |||
|
|
|||
There was a problem hiding this comment.
Can we add another code example here of how to use the partition option with parallel?
| if (partitions) { | ||
| partitions = partitions.join(','); | ||
| } | ||
| commandOptions.query = addToQuery(commandOptions.query, '_partitions', partitions); |
There was a problem hiding this comment.
We should leave this as _partition for consistency with the CLI params. QUnit will automatically treat repeated URL params as an array. So if we do tests?_partition=1&_partition=2 then you'll get QUnit.urlParams._partition === ['1', '2'].
| * | ||
| * @param {String} baseUrl | ||
| * @param {Array} partitions |
There was a problem hiding this comment.
Should be Array<Number> right?
| } | ||
| } | ||
|
|
||
|
|
Also fix a whitespace error and put a multi-partition/parallel example in the readme.
| @@ -34,7 +34,7 @@ module.exports = TestCommand.extend({ | |||
|
|
|||
| availableOptions: [ | |||
| { name: 'split', type: Number, description: 'A number of files to split your tests across.' }, | |||
| { name: 'partition', type: Number, description: 'The number of the partition to run after splitting.' }, | |||
| { name: 'partition', type: [Array, String], description: 'The number of the partition(s) to run after splitting.' }, | |||
There was a problem hiding this comment.
Shouldn't this be type: [Array, Number]?
| } | ||
|
|
||
| var group = partition - 1; | ||
| tests = tests.concat(lintTestGroups[group]).concat(otherTestGroups[group]); |
There was a problem hiding this comment.
concat() can accept multiple arrays at once, so let's do that instead.
| var group = partition - 1; | ||
| tests = tests.concat(lintTestGroups[group]).concat(otherTestGroups[group]); | ||
| } | ||
| return tests; |
There was a problem hiding this comment.
Nitpick: newline before this return.
|
@trentmwillis feedback addressed |
|
🎉 thanks for all your work on this! |
|
👍 |
|
@bendemboski @trentmwillis do you have an example of this working on circleci somewhere one could look at? I'm trying this in our circle setup but parallelizing tests within a container doesn't seem to make the overall test runs much faster. (happy to move this conversation to community slack instead) |
|
@Dhaulagiri I don't have an example that I can make public. My guess is that the limiting factor in your test speed is CPU, rather than blocking on async events or something, so parallelizing within the same container won't speed anything up. |
The primary use case is splitting test modules across multiple CI containers, and having each of those CI containers run its list of tests in multiple parallel runs (you would specify split, parallel, and multiple partitions for each container). This can also be used for unbalanced partitioning, e.g. if you want one container to run 2/3 of the tests and the other run 1/3 (you would specify split and then different numbers of partitions for different containers).
Fixes #62