Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/gzip causes deadlock when PipeReader is given. #13142

Closed
Maki-Daisuke opened this issue Nov 4, 2015 · 2 comments

Comments

Projects
None yet
3 participants
@Maki-Daisuke
Copy link

commented Nov 4, 2015

I tried to decompress gzip stream from pipe reader with compress/gzip, but it causes deadlock. Curious to say, the same code works well with compress/bzip2.

Here is PoC code:

package main

import (
    "bufio"
    "compress/gzip"
    "fmt"
    "io"
    "os"
)

func main() {
    rd, wr := io.Pipe()
    // unzipper := bzip2.NewReader(rd) // This works well in contrast.
    unzipper, err := gzip.NewReader(rd)
    if err != nil {
        panic(err)
    }
    go func() {
        file, err := os.Open("foo.gz")
        if err != nil {
            panic(err)
        }
        defer file.Close()
        io.Copy(wr, file)
        wr.Close()
    }()
    brd := bufio.NewReader(unzipper)
    for {
        line, err := brd.ReadString('\n')
        if err != nil {
            if err == io.EOF {
                break
            }
            panic(err)
        }
        fmt.Print(line)
    }
    fmt.Println("ok")
}

and, I got the following error message:

> go build ./cmd/temp && ./temp
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Syncsemacquire(0x820296040)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/runtime/sema.go:237 +0x201
sync.(*Cond).Wait(0x820296030)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/sync/cond.go:62 +0x9b
io.(*pipe).read(0x820296000, 0x82023b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/pipe.go:52 +0x2d2
io.(*PipeReader).Read(0x82024c020, 0x82023b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/pipe.go:134 +0x50
bufio.(*Reader).fill(0x820238120)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Read(0x820238120, 0x82029808d, 0xa, 0x200, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:207 +0x260
io.ReadAtLeast(0x88203cf2c8, 0x820238120, 0x82029808d, 0xa, 0x200, 0xa, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/io.go:298 +0xe6
io.ReadFull(0x88203cf2c8, 0x820238120, 0x82029808d, 0xa, 0x200, 0x0, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/io/io.go:316 +0x62
compress/gzip.(*Reader).readHeader(0x820298000, 0x107401, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/compress/gzip/gunzip.go:173 +0xe2
compress/gzip.NewReader(0x88203cf1f0, 0x82024c020, 0x1efa08, 0x0, 0x0)
    /Users/maki/.brew/Cellar/go/1.5.1/libexec/src/compress/gzip/gunzip.go:89 +0x207
main.main()
    /Users/maki/gocode/src/github.com/Maki-Daisuke/cmd/temp/main.go:14 +0x1ca

Yes, I used Homebrewed Go version 1.5.1 on Mac OS X (Yosemite). Should I test the code another environment?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Nov 4, 2015

gzip.NewReader tries to read the data from the pipe. At the time you call gzip.NewReader, there is nothing writing to the pipe and there are no other goroutines running. That is why you get that crash. The fix is to move the call to gzip.NewReader after you start the goroutine writing to the pipe.

@Maki-Daisuke

This comment has been minimized.

Copy link
Author

commented Nov 4, 2015

@ianlancetaylor:

Ah, I got it. Thank you so much!

@golang golang locked and limited conversation to collaborators Nov 4, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.