proposal: syscall: make use of StartupInfo.Reserved2 on Windows

[ZekeLu]

Background

This is a follow-up of #21085. The inheritance part of that issue has been addressed by #44011. But the discovery part remains. This proposal tries to address the discovery part of that issue.

The Proposal

On Windows, it's a de facto to use _get_osfhandle to discover the inherited file descriptors in the child process. For example, Chromium uses it to support its --remote-debugging-pipe option (see devtools_pipe_handler.cc). This C Runtime function depends on the information passed by the lpReserved2 field of the STARTUPINFOW struct. The MSDN just states that this field is Reserved for use by the C Run-time; must be NULL. Luckily, the implementation can be found at least in two projects:

• nodejs: https://github.com/nodejs/node/blob/71691e53/deps/uv/src/win/process-stdio.c#L32-L37
• ReactOS: https://github.com/reactos/reactos/blob/57c84dd6/sdk/lib/crt/stdio/file.c#L405-L532

As of now, the corresponding field in Go is omitted. So I propose to make use of this field to make the inherited file descriptors discoverable by the child process. Here is the change to the StartupInfo struct:

 type StartupInfo struct {
    Cb            uint32
    _             *uint16
    Desktop       *uint16
    Title         *uint16
    X             uint32
    Y             uint32
    XSize         uint32
    YSize         uint32
    XCountChars   uint32
    YCountChars   uint32
    FillAttribute uint32
    Flags         uint32
    ShowWindow    uint16
-   _             uint16
-   _             *byte
+   CbReserved2   uint16
+   Reserved2     *byte
    StdInput      Handle
    StdOutput     Handle
    StdErr        Handle
 }

I will send a CL to show the implementation later. If the CL is landed, the child process can get the inherited file descriptor like this (taking from the test in the upcoming CL):

func cmdCRTPipeHandle(args ...string) {
	get_osfhandle := syscall.NewLazyDLL("msvcrt.dll").NewProc("_get_osfhandle")

	h3, _, _ := get_osfhandle.Call(3)
	if h3 == uintptr(syscall.InvalidHandle) {
		fmt.Fprintf(os.Stderr, "_get_osfhandle: pipe 3 is invalid\n")
		os.Exit(1)
	}
	pipe3 := os.NewFile(h3, "in")
	defer pipe3.Close()

	h4, _, _ := get_osfhandle.Call(4)
	if h4 == uintptr(syscall.InvalidHandle) {
		fmt.Fprintf(os.Stderr, "_get_osfhandle: pipe 4 is invalid\n")
		os.Exit(1)
	}
	pipe4 := os.NewFile(h4, "out")
	defer pipe4.Close()

	// use the pipes, truncated
}

Credits

The idea is inspired by the comments in #21085. Thank you @zombiezen @alexbrainman @glasser @tv42 and all others who were taking part in the discussion!

[gopherbot]

Change https://go.dev/cl/415995 mentions this issue: syscall: make use of StartupInfo.Reserved2 on Windows