os/exec: Unable to close pipes properly if the caller can't close the pipes ahead

[wxiaoguang]

Go version

N/A

Output of go env in your module/workspace:

N/A

What did you do?

When using Cmd.Run to run a program, we need to make it serve, feed its input from another Goroutine, and we don't know the input ahead or when input will be closed.

A real case is git cat-file --batch-check.

• We need to start the process in background, and callers can send the object ID to its stdin, and read output to parse
• The git runs as a background service, the caller doesn't close the input ahead, and the caller don't know how many object IDs it needs to check ahead.
• But git cat-file --batch-check can exit immediacy if it runs in a non-git directory

func TestCmdPipe(t *testing.T) {
	inR, inW := io.Pipe()
	outR, outW := io.Pipe()
	errBuf := &bytes.Buffer{}
	
	_ = inW
	cmd := exec.Cmd{
		Path:   "/bin/bash", // the real case is `git cat-file --batch-check`
		Args:   []string{"/bin/bash", "no-such-script"},
		Stdin:  inR,
		Stdout: outW,
		Stderr: errBuf,
		// WaitDelay: time.Second, // does not help
	}

	err := cmd.Start()
	assert.NoError(t, err)

	// I know that inW.Close will make Wait exit, but we can't do that
	// because the inW is used in another goroutine and keeps querying the object IDs if the process runs

	err = cmd.Wait() // fatal error: all goroutines are asleep - deadlock!
	assert.NoError(t, err)

	_ = outR
}

What did you see happen?

Deadlock

• The inW goroutine doesn't know the process exits (Wait never returns), so it is unable to close the inW pipe
• The Wait keeps waiting because inW can't be closed, then deadlock

What did you expect to see?

Is it possible to know that the process has exited before Wait causes the deadlock?

If I can know that the process has exited in another Goroutine, I can manually close the inW, then everything goes well.

[seankhliao]

See os/exec.Cmd.Process: https://pkg.go.dev/os/exec#Cmd.Process

[wxiaoguang]

Maybe StdinPipe and StdoutPipe is the answer?

[wxiaoguang]

See os/exec.Cmd.Process: https://pkg.go.dev/os/exec#Cmd.Process

Hmm, I don't think Cmd.Process is the answer .....

[Jorropo]

You do not have to call cmd.Wait() in order to read from outR:

func TestCmdPipe(t *testing.T) {
	inR, inW := io.Pipe()
	outR, outW := io.Pipe()
	errBuf := &bytes.Buffer{}
	
	_ = inW
	cmd := exec.Cmd{
		Path:   "/bin/bash", // the real case is `git cat-file --batch-check`
		Args:   []string{"/bin/bash", "no-such-script"},
		Stdin:  inR,
		Stdout: outW,
		Stderr: errBuf,
		// WaitDelay: time.Second, // does not help
	}

	err := cmd.Start()
	assert.NoError(t, err)

	// I know that inW.Close will make Wait exit, but we can't do that
	// because the inW is used in another goroutine and keeps querying the object IDs if the process runs

	s := bufio.NewScanner(outR)
	for s.Scan() {
		_ = s.Text()
	}
	// error handling
}

The correct way to implement this is to have 2 goroutines, one writing to inW and one reading from outR.

[wxiaoguang]

You do not have to call cmd.Wait() in order to read from outR:

I didn't. That code is a demo

The correct way to implement this is to have 2 goroutines, one writing to inW and one reading from outR.

I already did that.

One goroutine for feeding inW, one goroutine for reading outR

The problem is:

• The inW goroutine doesn't know the process exits (Wait never returns), so it is unable to close the inW pipe
• The Wait keeps waiting because inW can't be closed, then deadlock

For example:

• Case 1: the inW goroutine finishes its job, then close, then Wait exits, then everything is fine.
• Case 2: the command itself exits due to error, then inW goroutine knows nothing about it, the Wait is still waiting for inW to be closed, then deadlock

[Jorropo]

Does @seankhliao's advice work ?

func TestCmdPipe(t *testing.T) {
	inR, inW := io.Pipe()
	outR, outW := io.Pipe()
	errBuf := &bytes.Buffer{}
	
	_ = inW
	cmd := exec.Cmd{
		Path:   "/bin/bash", // the real case is `git cat-file --batch-check`
		Args:   []string{"/bin/bash", "no-such-script"},
		Stdin:  inR,
		Stdout: outW,
		Stderr: errBuf,
		// WaitDelay: time.Second, // does not help
	}

	err := cmd.Start()
	assert.NoError(t, err)

	// I know that inW.Close will make Wait exit, but we can't do that
	// because the inW is used in another goroutine and keeps querying the object IDs if the process runs

	err = cmd.Process.Wait()
	assert.NoError(t, err)

	_ = outR
}

[wxiaoguang]

Does @seankhliao's advice work ?

Oh interesting! Maybe the low-level cmd.Process.Wait does work. I misunderstood that comment (I thought it meant to access that cmd.Process by other approaches, but indeed cmd.Process.Wait() seems good.

Thank you very much!

[wxiaoguang]

OK, after reading and writing more related code, I think I have some new and better understanding for this problem.

• I can use StdinPipe and StdoutPipe, but it makes code much more complicated than it should be.
• I can't use os/exec.Cmd.Process, because it is too difficult to use for daily work

See

• proposal: os/exec: support closing stdin/stdout pipes easily #77227