Windows anonymous pipes do not work with createReadStream

[mgirolet-gl]

Version

Tested with 22.14.0 and 20.18.1

Platform

Microsoft Windows NT 10.0.22000.0 x64

Subsystem

No response

What steps will reproduce the bug?

C# example for the anonymous pipe generator:

using System;
using System.Diagnostics;
using System.IO;
using System.IO.Pipes;
using System.Threading.Tasks;

namespace DotNetSide
{
    class Program
    {
        static async Task Main(string[] args)
        {
            using var pipeWriter = new AnonymousPipeServerStream(PipeDirection.Out, HandleInheritability.Inheritable);
            using var pipeReader = new AnonymousPipeServerStream(PipeDirection.In, HandleInheritability.Inheritable);

            Process client = new Process();

            client.StartInfo.FileName = "node";
            client.StartInfo.Arguments = "yourscript.js " + pipeWriter.GetClientHandleAsString() + " " + pipeReader.GetClientHandleAsString();
            client.StartInfo.UseShellExecute = false;
            client.Start();

            pipeWriter.DisposeLocalCopyOfClientHandle();
            pipeReader.DisposeLocalCopyOfClientHandle();

            _ = StartReadingAsync(pipeReader);

            using var sw = new StreamWriter(pipeWriter)
            {
                AutoFlush = true
            };

            string message = Console.ReadLine();

            while (message != "exit")
            {
                await sw.WriteAsync(message);
                message = Console.ReadLine();
            }

            client.Close();
        }

        private static async Task StartReadingAsync(AnonymousPipeServerStream pipeReader)
        {
            try
            {
                StreamReader sr = new StreamReader(pipeReader);

                while (true)
                {
                    var message = await sr.ReadLineAsync();

                    if (message != null)
                    {
                        Console.WriteLine(message);
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

JS code that should be able to read and write from the pipes' file descriptors:

const fs = require('fs');
const reader = fs.createReadStream(null, {fd: parseInt(process.argv[2], 10)});
const writer = fs.createWriteStream(null, {fd: parseInt(process.argv[3], 10)});

reader.on('data', data => writer.write('echo: ' + data + '\n'));

setInterval(()=> {}, 1000 * 60 * 60);

Source, should work according to the documentation and other examples I've seen.

How often does it reproduce? Is there a required condition?

Everytime I try to open an inherited anonymous pipe with NodeJS

What is the expected behavior? Why is that the expected behavior?

'data' events should be emitted when read pipes have new data, and it should be possible to send data to write pipes too.

What do you see instead?

Whenever I bind a callback to a ReadStream created from a pipe's FD:

node:events:496
      throw er; // Unhandled 'error' event
      ^

Error: EBADF: bad file descriptor, close
Emitted 'error' event on ReadStream instance at:
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4083,
  code: 'EBADF',
  syscall: 'close'
}

Whenever I try to write to a WriteStream created from a pipe's FD:

node:events:496
      throw er; // Unhandled 'error' event
      ^

Error: EBADF: bad file descriptor, close
Emitted 'error' event on WriteStream instance at:
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4083,
  code: 'EBADF',
  syscall: 'close'
}

Additional information

The same Anonymous pipe generator C# program given as an example works well with C++ using the Windows API, so it's not a C# problem:

#include <iostream>
#include <string>
#include <windows.h>
#include <array>

int main(int argc, const char** argv)
{
    std::string pipeHandleString = argv[1];
    int pipeHandleInt = std::stoi(pipeHandleString);

    HANDLE pipeHandle = (void*)pipeHandleInt;

    std::array<char, 256> buffer = {};

    DWORD numberOfBytesRead;

    BOOL result = false;

    while (result = ReadFile(pipeHandle, &buffer, 256, &numberOfBytesRead, nullptr)) {
        buffer[numberOfBytesRead] = '\0';
        std::cout << "echo: " << buffer.data() << std::endl;
    }
}

Of course, it also works with C# clients.

[mgirolet-gl]

After reading the nodejs source code for hours, I finally figured out the source of the problem!

Opening an anonymous pipe gives you a HANDLE directly, not a file descriptor.

However, UV (the dependency that handles various cross platform stuff for Node, like i/o) expects a an actual file descriptor, to be passed in uv__get_osfhandle which calls _get_osfhandle which is meant to turn a file descriptor into a HANDLE.

This of course doesn't work if you pass a HANDLE instead of a file descriptor, resulting in UV marking it as bad.

The way to work around this would be to call _open_osfhandle on the HANDLE, which does the opposite process, meaning it creates a file descriptor for a handle. Calling _get_osfhandle on the resulting file descriptor gives back the original HANDLE value.

This however doesn't seem to be possible to do with what we currently have: calling _open_osfhandle from my parent program that generates the anonymous pipe will give a file descriptor that can only be used by said parent program. It cannot be inherited by child processes as far as I could find. And I cannot call it from the Node process without editing Node's code which is obviously not doable for production purposes.

The solution would be for Node to directly allow inputting HANDLEs, or have UV try to read the pipe as a HANDLE if it failed reading it as a file descriptor.

[juanarbol]

ping @nodejs/libuv

[saghul]

The solution would be for Node to directly allow inputting HANDLEs

IIRC this is what we did for libuv v2...

[bnoordhuis]

It could be supported in v1.x by adding something like uv_pipe_open_ex.

uv_pipe_open literally calls HANDLE os_handle = uv__get_osfhandle(file); on its first line, from there on down it operates exclusively on handles.

[mgirolet-gl]

It could be supported in v1.x by adding something like uv_pipe_open_ex.

uv_pipe_open literally calls HANDLE os_handle = uv__get_osfhandle(file); on its first line, from there on down it operates exclusively on handles.

This, with a way for either Node or UV to know which one to call depending of whatever was passed to be opened, would solve the issue.

Any idea on the doability and ETA of this? Would I need to make a PR (please no) or are there node / libuv contributors who usually solve this kind of issue?

Thx to all of you for your quick replies, by the way :)

[saghul]

Any idea on the doability and ETA of this? Would I need to make a PR (please no) or are there node / libuv contributors who usually solve this kind of issue?

Doability seems guaranteed, it's a matter of someone rolling up their sleeves and sending the PR. Since you are the interested party here a good way to see it through is to take matters in your own hands.