Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

child_process: spawn() and spawnSync() validation #8312

Merged
merged 3 commits into from
Dec 25, 2016

Conversation

cjihrig
Copy link
Contributor

@cjihrig cjihrig commented Aug 28, 2016

Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
Affected core subsystem(s)

child_process, src

Description of change

This PR is intended to fix #8096. The first commit prevents the abort from #8096 by verifying that the handle type is correct before trying to close it. This commit is semver-patch.

This leaves an uninformative EINVAL error being thrown due to the bad input. So, the second commit in this PR addresses that by producing more helpful errors in the JavaScript layer. In doing so, it better unifies the spawn() and spawnSync() input checking (which also nicely bubbles up to the other child_process methods). Additional spawnSync() specific checks are also provided. This commit is semver-major due to the changes in error messages.

The third commit removes redundant C++ type checks that are now done in the JS layer.

cc: @bnoordhuis, @targos, and @retrohacker from #8096

Fixes: #8096
Fixes: #8539

@cjihrig cjihrig added child_process Issues and PRs related to the child_process subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. semver-major PRs that contain breaking changes and should be released in the next major version. labels Aug 28, 2016
// Validate the cwd, if present.
if (options.cwd !== undefined &&
options.cwd !== null &&
!typeof options.cwd === 'string')
Copy link
Contributor

@Fishrock123 Fishrock123 Aug 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a helper would be better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like string, null and undefined are allowed.

On Aug 29, 2016 12:09 AM, "Jeremiah Senkpiel" notifications@github.com
wrote:

In lib/child_process.js
#8312 (comment):

@@ -329,6 +329,49 @@ function normalizeSpawnArguments(file /, args, options/) {
else if (options === null || typeof options !== 'object')
throw new TypeError('"options" argument must be an object');

  • // Validate the cwd, if present.
  • if (options.cwd !== undefined &&
  •  options.cwd !== null &&
    
  •  !typeof options.cwd === 'string')
    

Maybe a helper would be better? won't !typeof options.cwd === 'string'
cover all these 3 cases?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nodejs/node/pull/8312/files/b4a6558fcbcf6cde50ff29af2d3487896ad91c68#r76536916,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABjB841u9wl3B9jlsDIb7RdsDKU0iWTcks5qkdXdgaJpZM4Ju-4I
.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean another function like isOptionalString() (where the optional part covers null and undefined)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!typeof options.cwd === 'string' is always false, it should be !(typeof options.cwd === 'string') or (better) typeof options.cwd !== 'string'.

@cjihrig
Copy link
Contributor Author

cjihrig commented Sep 1, 2016

@bnoordhuis any opinion on the approach here?

@bnoordhuis
Copy link
Member

The options.x !== undefined && options.x !== null clauses could be shortened to options.x != null and I'd probably put braces around the body of multi-line if statements.

Moving the validation logic to JS land basically LGTM though.

@bnoordhuis
Copy link
Member

@cjihrig Ping.

@cjihrig
Copy link
Contributor Author

cjihrig commented Oct 12, 2016

I plan on getting back to this next week. Sorry for the delay.

@rvagg rvagg force-pushed the master branch 2 times, most recently from c133999 to 83c7a88 Compare October 18, 2016 17:02
@cjihrig cjihrig force-pushed the spawnsync branch 2 times, most recently from af5c2f5 to 76d7181 Compare October 18, 2016 17:17
@cjihrig cjihrig force-pushed the spawnsync branch 3 times, most recently from 50f599e to 146129b Compare October 18, 2016 18:58
@cjihrig cjihrig changed the title WIP: child_process: spawn() and spawnSync() validation child_process: spawn() and spawnSync() validation Oct 18, 2016
@cjihrig
Copy link
Contributor Author

cjihrig commented Oct 18, 2016

@bnoordhuis updated, PTAL.

@cjihrig
Copy link
Contributor Author

cjihrig commented Oct 26, 2016

@bnoordhuis ping

@cjihrig
Copy link
Contributor Author

cjihrig commented Nov 15, 2016

ping @nodejs/collaborators

@sam-github
Copy link
Contributor

sam-github commented Nov 16, 2016

I'll look at this, but because the test was added in the same commit as the additional checks, its hard to evaluate what is different, i.e., what the behaviour used to be. If the test was added, then the commit that changed the behaviour changed the test... it would be easy to understand the change.

@sam-github
Copy link
Contributor

sam-github commented Nov 16, 2016

Many thumbs up for solid testing of these APIs, and better input validation.

cwd, detached, uid, gid are all implemented in normalizeSpawnArguments(), why aren't the tests in test/parallel/test-child-process-spawn-typeerror.js, instead of a new file?

Its not clear what tests should go where, and its also very difficult to determine what kind of test coverage there is, not in the line sense, but in the sense of: are the tests against spawnSync here valid for any of the other Sync calls? Any other spawn calls?

Its painful, but maybe the tests should be factored so they can be applied to all the child_process functions that use the same normalize? Or perhaps the tests should all just be in one file, with a comment in the test saying that even though it appears that only one API is tested, the underlying code is the same, so its hitting every API?

Unrelated to this PR, but I'm pretty disappointed that the arg permutation tests in test-child-process-spawn-typeerror.js were never extended to include *Sync, since arg permutations was such a rich source of bugs and inconsistencies, and I'm poking around, but I can't see any tests for their arg permutations.

But even though that problem is unrelated, its why I suggest that having the tests be organized in a way that its (EDIT) "easy" for a reader to verify the coverage scenarios is important.


// Validate windowsVerbatimArguments, if present.
if (options.windowsVerbatimArguments != null &&
typeof options.windowsVerbatimArguments !== 'boolean') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be, literally, boolean? I would expect it to have to be truthy/falsy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could probably be coerced to a Boolean, but the extra strictness is good IMO. Not that it matters much, but windowsVerbatimArguments isn't documented anyway, IIRC.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, its internal-only, so maybe not so important, and better to start strict. I don't agree strictness is generally good in this case, btw. It creates a bizarre API wherein truthy is sometimes true/false, sometimes null/undefined/0/"" are also false, sometimes not, and sometimes function/[]/{}/non-zero/"strings" are also true, and sometimes null or undefined are a "third way", neither true or false but some kind of other behaviour. Ouch. Anyhow, as a semi-private (but used by modules on npmjs.org none-the-less) API, I'm OK with this restriction.

@cjihrig
Copy link
Contributor Author

cjihrig commented Nov 16, 2016

cwd, detached, uid, gid are all implemented in normalizeSpawnArguments(), why aren't the tests in test/parallel/test-child-process-spawn-typeerror.js, instead of a new file?

I just did that because test-child-process-spawn-typeerror validates the types of the arguments passed in, but not specific properties (is options an object, but not specific values in options).

Its not clear what tests should go where, and its also very difficult to determine what kind of test coverage there is, not in the line sense, but in the sense of: are the tests against spawnSync here valid for any of the other Sync calls?

I agree that the tests are kind of all over the place. This PR is a step towards consolidating the behavior of spawn() and spawnSync(). Consolidating the tests would be a good followup.

Its painful, but maybe the tests should be factored so they can be applied to all the child_process functions that use the same normalize?

I think just about everything goes through normalizeSpawnArguments(), which is convenient.

@sam-github
Copy link
Contributor

I just did that because test-child-process-spawn-typeerror validates the types of the arguments passed in, but not specific properties (is options an object, but not specific values in options).

The tests you add seemed like an obvious candidate to be in a "spawn typeerror" test, but I understand the distinction you draw above.

If you do draw that distinction, my request is that the name of the tests (and a comment at their top) clearly communicate it: the names/comments should indicate what that test is for, so that we don't get diffusion and confusion of tests over time.

For example:

  • test-child-process-spawn-typeerror could perhaps be test-child-process-argument-permutations (with an XXX comment pointing out that the *Sync APIs are currently untested for permutation).
  • test-child-process-spawnsync-validation-errors.js could be test-child-process-validation-errors.js (with a comment pointing out that while it doesn't explicitly test all the child_process APIs, that the underlying code is currently the same so they all inherit the same behaviour - or else just explicitly test them all so the tests are robust in the face of future refactors).

I suggest a rename like this because it would be a disaster if someone created a test-child-process-spawn-validation-errors.js (note the lack of sync) that had different validations in it than the spawnsync version you just made... the validations need to be in one place, so that the tests enforce consistent behaviour across the child_process API. For example, uid shouldn't be asserted to be integral for one API, but just a Number for another, and to be any Number except NaN for yet a third (which was the kind of situation we used to have).


pass('uid', undefined);
pass('uid', null);
pass('uid', process.getuid());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert NaN is not valid. Ditto below.

pass('shell', undefined);
pass('shell', null);
pass('shell', false);
fail('shell', 0, err);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 and 1 are usually acceptable as truthy values, I think they should be accepted here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally go for strict.

Copy link
Member

@bnoordhuis bnoordhuis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Good test coverage.

pass('shell', undefined);
pass('shell', null);
pass('shell', false);
fail('shell', 0, err);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally go for strict.

@@ -315,6 +315,9 @@ function _convertCustomFds(options) {
function normalizeSpawnArguments(file /*, args, options*/) {
var args, options;

if (typeof file !== 'string' || file.length === 0)
throw new TypeError('"file" argument must be a string');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: If the input is an empty string, the error message might not be exactly appropriate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to specify "non-empty string"

@cjihrig
Copy link
Contributor Author

cjihrig commented Dec 22, 2016

@cjihrig
Copy link
Contributor Author

cjihrig commented Dec 23, 2016

CI is all green.

This commit verifies that the child process handle is of the
correct type before trying to close it in
CloseHandlesAndDeleteLoop(). This catches the case where input
validation failed, and the child process was never actually
spawned.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit applies stricter input validation in
normalizeSpawnArguments(), which is run by all of the
child_process methods. Additional checks are added for spawnSync()
specific inputs.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit removes C++ checks from spawn() and spawnSync()
that are duplicates of the JavaScript type checking.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@cjihrig
Copy link
Contributor Author

cjihrig commented Dec 25, 2016

Landed in b374ee8 (semver patch), fc7b0dd, and 45c9ca7.

@MylesBorins
Copy link
Contributor

@Fishrock123 the bot should likely not be adding lts labels to anything labelled semver major

@Fishrock123
Copy link
Contributor

italoacasas pushed a commit to italoacasas/node that referenced this pull request Jan 18, 2017
This commit verifies that the child process handle is of the
correct type before trying to close it in
CloseHandlesAndDeleteLoop(). This catches the case where input
validation failed, and the child process was never actually
spawned.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
italoacasas pushed a commit to italoacasas/node that referenced this pull request Jan 18, 2017
This commit applies stricter input validation in
normalizeSpawnArguments(), which is run by all of the
child_process methods. Additional checks are added for spawnSync()
specific inputs.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
italoacasas pushed a commit to italoacasas/node that referenced this pull request Jan 18, 2017
This commit removes C++ checks from spawn() and spawnSync()
that are duplicates of the JavaScript type checking.

Fixes: nodejs#8096
Fixes: nodejs#8539
Refs: nodejs#9722
PR-URL: nodejs#8312
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@jasnell jasnell mentioned this pull request Apr 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. child_process Issues and PRs related to the child_process subsystem. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
10 participants