os/exec: Piping to a command blocks under freebsd when input string has length >= 8192

[AlexanderThaller]

1. Go Version: go version go1.4 freebsd/amd64
2. OS Version/Architecture: 10.0-RELEASE-p6 FreeBSD 10.0-RELEASE-p6 #0: Tue Jun 24 07:47:37 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
3. I'm trying to pipe arbitrary strings to an external command which sometimes hangs for some reason.
4. I want to get the CombinedOutput of the command after executing.
5. The application hangs while piping.

I investigated and found out that the piping to the command won't finish sometimes. I wrote a program to check under which circumstances the piping would hang and found out that if the string is 8192 chars long (or longer) the piping would hang. Here is the program I used which just pipes to cat and uses increasing sizes for the string:

package main

import (
    "io"
    "log"
    "math/rand"
    "os/exec"
)

var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")

func randSeq(n int) string {
    b := make([]rune, n)
    for i := range b {
        b[i] = letters[rand.Intn(len(letters))]
    }
    return string(b)
}

func main() {
    for i := 8190; i < 8195; i++ {
        log.Println("length: ", i)
        data := randSeq(i)

        command := exec.Command("/bin/cat")

        pipe, err := command.StdinPipe()
        if err != nil {
            log.Fatal("can not get pipe from command: ", err)
        }

        log.Println("writing to pipe")
        _, err = io.WriteString(pipe, data)
        if err != nil {
            log.Fatal("can not write to pipe: ", err)
        }
        log.Println("finished writing to pipe")
        pipe.Close()

        _, err = command.CombinedOutput()
        if err != nil {
            log.Fatal("problem while running command: ", err)
        }

    }
}

I tried the same under Linux amd64 with the same go version and the program worked as expected.

[AlexanderThaller]

When I do the same in the shell the command will execute as expected:

$ openssl rand -base64 8192 | tr -d '\n' | cat

[minux]

Your program is expecting that you can write a pipe 8KB worth of data
without blocking, but that's not universally true on Unixes.

For example, if I run your program on Linux and change to write 128KB of
data to the pipe, the program will also block.

The reason your shell command works is that the cat process is running and
consuming content from the pipe concurrently.

Your Go program, however, doesn't start the cat process until the whole
WriteString is completed.

[minux]

Closing as working as intended.