Reverse proxy with gzip generates unexpected EOFs when reading body.

[optimality]

I'm trying to gzip the response from an API server that uses reverse proxies to send requests to a number of other servers. However, whenever I try to gzip the results, reading the body generates io.ErrUnexpectedEOF. I suspect there's a bug in how ReverseProxy handles the gzip writer.

1. What version of Go are you using (go version)?
   go version go1.6 darwin/amd64
2. What operating system and processor architecture are you using (go env)?
   darwin amd64 (OS X 10.11.3)
3. What did you do?
   If possible, provide a recipe for reproducing the error.
   A complete runnable program is good.
   A link on play.golang.org is best.

Demo code:

package main

import (
    "compress/gzip"
    "fmt"
    "io"
    "io/ioutil"
    "net/http"
    "net/http/httputil"
    "net/url"
    "strings"
    "time"
)

// Gzip from https://gist.github.com/the42/1956518
type gzipResponseWriter struct {
    io.Writer
    http.ResponseWriter
}

func (w gzipResponseWriter) Write(b []byte) (int, error) {
    return w.Writer.Write(b)
}

func makeGzipHandler(fn http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
            fn(w, r)
            return
        }
        w.Header().Set("Content-Encoding", "gzip")
        gz := gzip.NewWriter(w)
        defer gz.Close()
        gzr := gzipResponseWriter{Writer: gz, ResponseWriter: w}
        fn(gzr, r)
    }
}

// Handler that does not set a content length, so, golang uses chunking.
func handler(w http.ResponseWriter, r *http.Request) {
    message := "Hello, world!"
    w.Header().Set("Content-Type", "text/plain")
    w.Write([]byte(message))
}

// Constructs a reverse proxy to the given port.
func reverseProxy(port string) func(http.ResponseWriter, *http.Request) {
    url, err := url.Parse("http://127.0.0.1" + port)
    if err != nil {
        panic(err)
    }
    return httputil.NewSingleHostReverseProxy(url).ServeHTTP
}

// Gets the content from the given server, then returns the error from reading the body.
func get(server http.Server) error {
    resp, err := http.Get("http://127.0.0.1" + server.Addr)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    _, err = ioutil.ReadAll(resp.Body)
    return err
}

func main() {
    server := http.Server{
        Addr:    ":2000",
        Handler: http.HandlerFunc(handler),
    }
    go server.ListenAndServe()

    proxyServer := http.Server{
        Addr:    ":4000",
        Handler: makeGzipHandler(reverseProxy(server.Addr)),
    }
    go proxyServer.ListenAndServe()

    time.Sleep(10 * time.Millisecond)

    fmt.Printf("Server err: %v\n", get(server))
    fmt.Printf("Proxy server err: %v\n", get(proxyServer))
}

1. What did you expect to see?
   No errors.
2. What did you see instead?
   Error when reading proxied response.

[optimality]

Note that I can also reproduce the response failures with curl, but only when asking for a compressed response:

% curl http://127.0.0.1:2000 -i
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 28 Mar 2016 18:52:23 GMT
Content-Length: 13

% curl http://127.0.0.1:4000 -i
HTTP/1.1 200 OK
Content-Length: 13
Content-Type: text/plain
Date: Mon, 28 Mar 2016 18:52:25 GMT

Hello, world!%

% curl http://127.0.0.1:2000 -i --compressed
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 28 Mar 2016 18:52:32 GMT
Content-Length: 13

Hello, world!%

% curl http://127.0.0.1:4000 -i --compressed
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 13
Content-Type: text/plain
Date: Mon, 28 Mar 2016 18:52:38 GMT

curl: (18) transfer closed with 3 bytes remaining to read

[ghost]

You didn't check errors from gz.Close(). Here's an improved example: http://play.golang.org/p/Fc_OZcq-PK
Output:

Server err: <nil>
2016/04/02 18:12:16 gz.Close: Conn.Write wrote more than the declared Content-Length
exit status 1

The content length field is set here: server.go

[optimality]

@opennota Great catch, thanks! I fixed this with:

func (w gzipResponseWriter) WriteHeader(code int) {
    w.Header().Del("Content-Length")
    w.ResponseWriter.WriteHeader(code)
}

Do you think there's a better fix?

[bradfitz]

@optimality, that seems good.

[Romanowiczmarek]

Does it work? I don't think WriteHeader(code int) is actually called in this case although it should be.

[optimality]

Yeah, this definitely fixed it, and WriteHeader is definitely called. Check out the following example code:

package main

import (
    "compress/gzip"
    "fmt"
    "io"
    "io/ioutil"
    "net/http"
    "net/http/httputil"
    "net/url"
    "strings"
    "time"
)

// Gzip from https://gist.github.com/the42/1956518
type gzipResponseWriter struct {
    io.Writer
    http.ResponseWriter
}

func (w gzipResponseWriter) Write(b []byte) (int, error) {
    return w.Writer.Write(b)
}

func (w gzipResponseWriter) WriteHeader(code int) {
    fmt.Printf("Writing header: %v\n", code)
    w.Header().Del("Content-Length")
    w.ResponseWriter.WriteHeader(code)
}

func makeGzipHandler(fn http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
            fn(w, r)
            return
        }
        w.Header().Set("Content-Encoding", "gzip")
        gz := gzip.NewWriter(w)
        defer func() {
            err := gz.Close()
            if err != nil {
                fmt.Printf("Error closing gzip: %+v\n", err)
            }
        }()
        gzr := gzipResponseWriter{Writer: gz, ResponseWriter: w}
        fn(gzr, r)
    }
}

// Handler that does not set a content length, so, golang uses chunking.
func handler(w http.ResponseWriter, r *http.Request) {
    message := "Hello, world!"
    w.Header().Set("Content-Type", "text/plain")
    w.Write([]byte(message))
}

// Constructs a reverse proxy to the given port.
func reverseProxy(port string) func(http.ResponseWriter, *http.Request) {
    url, err := url.Parse("http://127.0.0.1" + port)
    if err != nil {
        panic(err)
    }
    return httputil.NewSingleHostReverseProxy(url).ServeHTTP
}

// Gets the content from the given server, then returns the error from reading the body.
func get(server http.Server) error {
    resp, err := http.Get("http://127.0.0.1" + server.Addr)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    _, err = ioutil.ReadAll(resp.Body)
    return err
}

func main() {
    server := http.Server{
        Addr:    ":2000",
        Handler: http.HandlerFunc(handler),
    }
    go server.ListenAndServe()

    proxyServer := http.Server{
        Addr:    ":4000",
        Handler: makeGzipHandler(reverseProxy(server.Addr)),
    }
    go proxyServer.ListenAndServe()

    time.Sleep(10 * time.Millisecond)

    fmt.Printf("Server err: %v\n", get(server))
    fmt.Printf("Proxy server err: %v\n", get(proxyServer))
}