compress/gzip causes deadlock when PipeReader is given.

[Maki-Daisuke]

I tried to decompress gzip stream from pipe reader with compress/gzip, but it causes deadlock. Curious to say, the same code works well with compress/bzip2.

Here is PoC code:

package main

import (
    "bufio"
    "compress/gzip"
    "fmt"
    "io"
    "os"
)

func main() {
    rd, wr := io.Pipe()
    // unzipper := bzip2.NewReader(rd) // This works well in contrast.
    unzipper, err := gzip.NewReader(rd)
    if err != nil {
        panic(err)
    }
    go func() {
        file, err := os.Open("foo.gz")
        if err != nil {
            panic(err)
        }
        defer file.Close()
        io.Copy(wr, file)
        wr.Close()
    }()
    brd := bufio.NewReader(unzipper)
    for {
        line, err := brd.ReadString('\n')
        if err != nil {
            if err == io.EOF {
                break
            }
            panic(err)
        }
        fmt.Print(line)
    }
    fmt.Println("ok")
}

and, I got the following error message:

> go build ./cmd/temp && ./temp
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Syncsemacquire(0x820296040)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/runtime/sema.go:237 +0x201
sync.(*Cond).Wait(0x820296030)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/sync/cond.go:62 +0x9b
io.(*pipe).read(0x820296000, 0x82023b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/pipe.go:52 +0x2d2
io.(*PipeReader).Read(0x82024c020, 0x82023b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/pipe.go:134 +0x50
bufio.(*Reader).fill(0x820238120)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Read(0x820238120, 0x82029808d, 0xa, 0x200, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:207 +0x260
io.ReadAtLeast(0x88203cf2c8, 0x820238120, 0x82029808d, 0xa, 0x200, 0xa, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/io.go:298 +0xe6
io.ReadFull(0x88203cf2c8, 0x820238120, 0x82029808d, 0xa, 0x200, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/io.go:316 +0x62
compress/gzip.(*Reader).readHeader(0x820298000, 0x107401, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/compress/gzip/gunzip.go:173 +0xe2
compress/gzip.NewReader(0x88203cf1f0, 0x82024c020, 0x1efa08, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/compress/gzip/gunzip.go:89 +0x207
main.main()
    /Users/maki/gocode/src/github.com/Maki-Daisuke/cmd/temp/main.go:14 +0x1ca

Yes, I used Homebrewed Go version 1.5.1 on Mac OS X (Yosemite). Should I test the code another environment?

[ianlancetaylor]

gzip.NewReader tries to read the data from the pipe. At the time you call gzip.NewReader, there is nothing writing to the pipe and there are no other goroutines running. That is why you get that crash. The fix is to move the call to gzip.NewReader after you start the goroutine writing to the pipe.

[Maki-Daisuke]

@ianlancetaylor:

Ah, I got it. Thank you so much!