Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limited Process Pools #791

Merged
merged 14 commits into from
May 12, 2016
Merged

Limited Process Pools #791

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
94dd695
Early POC of limited process pools. Initial attempt at addressing #718
dcousineau Apr 26, 2016
c646eb0
Cleaned up experimental concurrency pool, renamed CLI flag.
dcousineau Apr 28, 2016
bc3a9f6
Merge branch 'master' into process-pool
dcousineau Apr 28, 2016
18ab69d
Cleaned up error cases and blank results handling
dcousineau Apr 28, 2016
d1615a8
Formatting CLI flags with minimum of two spaces
dcousineau Apr 28, 2016
c6bc602
Updated help output and README
dcousineau Apr 28, 2016
9f58c09
Merge branch 'master' into process-pool
dcousineau May 2, 2016
0371cd1
Remove superfluous output
dcousineau May 2, 2016
6bfbe19
Merge branch 'master' into process-pool
dcousineau May 5, 2016
e4a7c85
Fixed --match handling for process pools, cleaned up limited pool run…
dcousineau May 5, 2016
cb996bc
Strings to single quote, fixing xo error
dcousineau May 5, 2016
24fddb1
Looser test title matching for varying Windows output
dcousineau May 5, 2016
631c9bf
Consolodated exception handling and cleaned up reporting code
dcousineau May 7, 2016
777d405
Additional testfiles loaded in certain testcases to ensure greater th…
dcousineau May 9, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 94 additions & 9 deletions api.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,19 @@ Api.prototype._run = function (files, _options) {
self.precompiler = new CachingPrecompiler(cacheDir, self.options.babelConfig);
self.fileCount = files.length;

var overwatch;
if (this.options.concurrency > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still use _runNoPool if concurrency is less than the fileCount.

Copy link
Contributor Author

@dcousineau dcousineau May 7, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally I would agree but with the current behavior, no matter what if the --concurrency flag is used the code behind _runLimitedPool is used. I vote we keep this as-is until we decide to drop _runNoPool entirely so that we can deterministically say if you use the concurrency flag you will use the new process pool code.

If enough people object to my opinion, however, I will change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if I add a concurrency option to package.json, I want to be able to re-enable .only behavior. By doing

ava [files ...]

Where the number of files is less than concurrency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm similarly worried about people using the --concurrency flag and seeing different behaviors based on their filename glob patterns. It's much simpler to convey in documentation "If the concurrency flag/option is present, automatic .only behavior is not supported" than a list of rules.

Additionally, a user manually running ava with a deliberate subset of tests is likely only doing so to run a few specific tests because they internally do not use .only (like my team) which actually cycles back up to my argument of nixing .only entirely since with the --watch flag and good file name globbing, there is no real need for .only.

overwatch = this._runLimitedPool(files, runStatus, self.options.serial ? 1 : this.options.concurrency);
} else {
// _runNoPool exists to preserve legacy behavior, specifically around `.only`
overwatch = this._runNoPool(files, runStatus);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment that _runNoPool is used to preserve .only behavior?

}

return overwatch;
};

Api.prototype._runNoPool = function (files, runStatus) {
var self = this;
var tests = new Array(self.fileCount);

// TODO: thid should be cleared at the end of the run
Expand Down Expand Up @@ -154,15 +167,7 @@ Api.prototype._run = function (files, _options) {
file: path.relative('.', file)
});

return {
stats: {
passCount: 0,
skipCount: 0,
todoCount: 0,
failCount: 0
},
tests: []
};
return getBlankResults();
});
}));
}
Expand Down Expand Up @@ -224,3 +229,83 @@ Api.prototype._run = function (files, _options) {
return runStatus;
});
};

function getBlankResults() {
return {
stats: {
testCount: 0,
passCount: 0,
skipCount: 0,
todoCount: 0,
failCount: 0
},
tests: []
};
}

Api.prototype._runLimitedPool = function (files, runStatus, concurrency) {
var self = this;
var tests = {};

runStatus.on('timeout', function () {
Object.keys(tests).forEach(function (file) {
var fork = tests[file];
fork.exit();
});
});

return Promise.map(files, function (file) {
var handleException = function (err) {
runStatus.handleExceptions({
exception: err,
file: path.relative('.', file)
});
};

try {
var test = tests[file] = self._runFile(file, runStatus);

return new Promise(function (resolve, reject) {
var runner = function () {
self.emit('ready');
var options = {
// If we're looking for matches, run every single test process in exclusive-only mode
runOnlyExclusive: self.options.match.length > 0
};
test.run(options)
.then(resolve)
.catch(reject);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.then(resolve, reject). resolve won't throw, so we can save the extra promise allocation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meh, I prefer it the existing way. Two argument then is considered an anti pattern now.

};

test.on('stats', runner);
test.on('exit', function () {
delete tests[file];
});
test.catch(runner);
}).catch(handleException);
} catch (err) {
handleException(err);
}
}, {concurrency: concurrency})
.then(function (results) {
// Filter out undefined results (usually result of caught exceptions)
results = results.filter(Boolean);

// cancel debounced _onTimeout() from firing
if (self.options.timeout) {
runStatus._restartTimer.cancel();
}

if (self.options.match.length > 0 && !runStatus.hasExclusive) {
// Ensure results are empty
results = [];
runStatus.handleExceptions({
exception: new AvaError('Couldn\'t find any matching tests'),
file: undefined
});
}

runStatus.processResults(results);
return runStatus;
});
};
32 changes: 18 additions & 14 deletions cli.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,17 +60,18 @@ var cli = meow([
' ava [<file|directory|glob> ...]',
'',
'Options',
' --init Add AVA to your project',
' --fail-fast Stop after first test failure',
' --serial, -s Run tests serially',
' --require, -r Module to preload (Can be repeated)',
' --tap, -t Generate TAP output',
' --verbose, -v Enable verbose output',
' --no-cache Disable the transpiler cache',
' --match, -m Only run tests with matching title (Can be repeated)',
' --watch, -w Re-run tests when tests and source files change',
' --source, -S Pattern to match source files so tests can be re-run (Can be repeated)',
' --timeout, -T Set global timeout',
' --init Add AVA to your project',
' --fail-fast Stop after first test failure',
' --serial, -s Run tests serially',
' --require, -r Module to preload (Can be repeated)',
' --tap, -t Generate TAP output',
' --verbose, -v Enable verbose output',
' --no-cache Disable the transpiler cache',
' --match, -m Only run tests with matching title (Can be repeated)',
' --watch, -w Re-run tests when tests and source files change',
' --source, -S Pattern to match source files so tests can be re-run (Can be repeated)',
' --timeout, -T Set global timeout',
' --concurrency, -c Maximum number of test files running at the same time (EXPERIMENTAL)',
'',
'Examples',
' ava',
Expand All @@ -88,7 +89,8 @@ var cli = meow([
'require',
'timeout',
'source',
'match'
'match',
'concurrency'
],
boolean: [
'fail-fast',
Expand All @@ -106,7 +108,8 @@ var cli = meow([
m: 'match',
w: 'watch',
S: 'source',
T: 'timeout'
T: 'timeout',
c: 'concurrency'
}
});

Expand All @@ -133,7 +136,8 @@ var api = new Api({
explicitTitles: cli.flags.watch,
match: arrify(cli.flags.match),
babelConfig: conf.babel,
timeout: cli.flags.timeout
timeout: cli.flags.timeout,
concurrency: cli.flags.concurrency ? parseInt(cli.flags.concurrency, 10) : 0
});

var reporter;
Expand Down
23 changes: 12 additions & 11 deletions readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,17 +136,18 @@ $ ava --help
ava [<file|directory|glob> ...]

Options
--init Add AVA to your project
--fail-fast Stop after first test failure
--serial, -s Run tests serially
--require, -r Module to preload (Can be repeated)
--tap, -t Generate TAP output
--verbose, -v Enable verbose output
--no-cache Disable the transpiler cache
--match, -m Only run tests with matching title (Can be repeated)
--watch, -w Re-run tests when tests and source files change
--source, -S Pattern to match source files so tests can be re-run (Can be repeated)
--timeout, -T Set global timeout
--init Add AVA to your project
--fail-fast Stop after first test failure
--serial, -s Run tests serially
--require, -r Module to preload (Can be repeated)
--tap, -t Generate TAP output
--verbose, -v Enable verbose output
--no-cache Disable the transpiler cache
--match, -m Only run tests with matching title (Can be repeated)
--watch, -w Re-run tests when tests and source files change
--source, -S Pattern to match source files so tests can be re-run (Can be repeated)
--timeout, -T Set global timeout
--concurrency, -c Maximum number of test files running at the same time (EXPERIMENTAL)

Examples
ava
Expand Down
Loading