/dev/stdin doesn't play well with child.stdin.write in 0.7/0.8-pre

[andris9]

When using /dev/stdin as an input file in child process and feeding data to it through child.stdin.write() the child returns an error

{ [Error: UNKNOWN, open '/dev/stdin'] errno: -1, code: 'UNKNOWN', path: '/dev/stdin' }

This only applies to 0.7/0.8-pre branch but not to 0.6 which works fine. The latest version of node I was using was bc73abe

I'm aware about a recent fix for /dev/stdin 1d3d02c but this only resolved when using stdin directly (eg. throug terminal and ending with Ctrl+D) but not when using stdin.write() from master node process.

Sample

Master process (master.js)

var spawn = require("child_process").spawn,
    test = spawn("node", [__dirname + "/child.js"]);

test.stdin.write("Hello world!");
test.stdin.end();

test.stdout.on("data", function(data){
    console.log(data.toString());
});

Child process (child.js)

var fs = require("fs");

fs.readFile("/dev/stdin", "utf-8", function(error, callback){
    console.log(error || callback);
});

When running node master.js in 0.6 the scripts outputs "Hello world!" but in 0.7/0.8-pre it outputs an error.

[bnoordhuis]

The open() syscall fails with ENXIO which isn't mapped in libuv because you should never see that error in the first place.

Why are you opening /dev/stdin? It's not as if the child process has a real stdin attached to it.

[andris9]

Actually I do not want to open /dev/stdin but to use it as input source for an another program and then feed the data through child.stdin.write()

I want to create a CSR with openssl and the command takes a keyfile as a parameter, I have the key as a string and do not want to write it to disk and later delete it, so instead I define the input file as /dev/stdin and write the key to the openssl process with stdin.write(). The actual command looks smth. like the following:

 openssl req -new -sha1 -subj /CN=localhost -key /dev/stdin

And I open it with child_process.spawn

var openssl = spawn("openssl", ["req", "-new", "-sha1", "-subj", 
                                      "/CN=localhost", "-key", "/dev/stdin"]);
openssl.stdin.write(keyString);
openssl.stdin.end();
openssl.stdout.on("data", console.log);

Everything works fine with node v0.6 but fails with an error with node v0.7 and up.

[bnoordhuis]

Okay, I see your point. Here is what happens:

In v0.4, file descriptors 0-2 were UNIX sockets (so you can send file descriptors to the child process). In v0.6, they were pipes. In v0.8, they're sockets again.

To wit:

var spawn = require('child_process').spawn;
var proc = spawn('ls', ['-l', '/proc/self/fd/0']);
proc.stdout.pipe(process.stdout);
proc.stderr.pipe(process.stderr);

Prints something like this:

$ v0.6/out/Release/node stdin.js
lr-x------ 1 bnoordhuis bnoordhuis 64 2012-06-25 23:34 /proc/self/fd/0 -> pipe:[207120066]

$ v0.8/out/Release/node stdin.js
lrwx------ 1 bnoordhuis bnoordhuis 64 2012-06-25 23:34 /proc/self/fd/0 -> socket:[207120017]

Linux (and probably most Unices) won't let you open /dev/stdin if it's a socket. You can work around that by rewriting your command like this:

var openssl = spawn("sh", ["-c",
  "cat | openssl req -new -sha1 -subj /CN=localhost -key /dev/stdin"]);

That turns the /dev/stdin that openssl sees into a regular pipe.

It's unfortunate that the change from pipes to sockets breaks some existing applications. On the one hand, it's a regression. On the other hand, the new child process implementation is dramatically better than the old one. I'll have to think about how (or if) to address this issue.

[andris9]

Thanks for the explanation! The thing seemed mysterious but when regarding pipes vs. sockets it became a lot clearer.

I think this resolves my particular problem, so feel free to close this issue.

[bnoordhuis]

Okay, thanks for the understanding. :) I'll close the issue then.

[Qix-]

@bnoordhuis are they still sockets as of 0.12? Is that a frozen implementation?

[bnoordhuis]

Yes, they're still UNIX sockets. I think you can say it's de facto frozen now, it's part of the libuv API. The UV_CREATE_PIPE flag to uv_spawn() actually creates a UNIX socketpair.

[Qix-]

Perfect, thanks!

[jeromew]

@bnoordhuis, I encountered this problem with the sqlite cli & inkscape cli.
I have been trying to find a way to make this work that does not use the sh -c cat | .. trick that you gave above.

the npm posix-pipe module makes it possible to obtain pipe-based fd (not socket based) and the following mostly works

var spawn = require('child_process').spawn;
var pipe = require('posix-pipe');
var streams = pipe();
var proc = spawn('cat',['/dev/stdin'], { stdio: [streams[0], 'pipe', 'pipe']});

proc.stdout.pipe(process.stdout);
proc.stderr.pipe(process.stderr);

streams[1].end('Hello World');

It is correctly piping Hello World. The only problem I have now is that I cannot find a way to make the process stop. streams[1] emits 'finish', but somehow streams[0] does not 'end'.

When doing

var proc = spawn('ls', ['-l', '/proc/self/fd/0'], { stdio: [streams[0], 'pipe', 'pipe']});
I get

lr-x------. 1 jwagner jwagner 64 Aug 27 19:33 /proc/self/fd/0 -> pipe:[450344]

and the process stops. streams[0] does not 'end' either.

Any help would be appreciated

[addaleax]

@jeromew This issue tracker is largely abandoned, for questions like these you can always go to https://github.com/nodejs/help.

What you are seeing here is an issue with the posix-pipe module, and not with Node core; please create a new bug report there: https://github.com/djphoenix/posix-pipe. Concretely, posix-pipe doesn’t set the O_CLOEXEC flag on the pipe socket descriptors, so when you spawn cat the pipe is actually duplicated into the child process, where it remains open. That keeps the other end of the pipe open in the parent process.

[jeromew]

@addaleax Thanks a lot ! I did not realize I was in a old issue tracker sorry.

I re-compiled posix-pipe with the flag you mentioned and it works like a charm ! I will send a PR to the owner of the module.

Thank you very much for your time on this