Skip to content

net/http: http.Client connection reuse behaviour differs when receiving from HTTP response body with Content-Encoding: gzip vs. uncompressed #8910

Closed
@gopherbot

Description

@gopherbot

by akrennmair:

What does 'go version' print?

go version go1.3.3 darwin/amd64

What steps reproduce the problem?
If possible, include a link to a program on play.golang.org.

1. go run server.go (see attached files)
2. go run client.go 
3. use tcpdump to observe that one connection is reused
4. go run client.go -compressed
5. use tcpdump to observe that for each HTTP request, a new connection is opened and
then closed again.

What happened?

In client.go, the resp.Body.Read() read the same content from the HTTP response body but
returned different values (err == io.EOF for uncompressed vs. err == nil for
compressed). In the case of the compressed HTTP response body, connection reuse did not
work properly.

What should have happened instead?

The resp.Body.Read() should have behaved the same, no matter whether the underlying HTTP
response is Content-Encoding: gzip or not, and connection reuse should have worked in
both cases.

Please provide any additional information below.

What I could gather from reading the source, http.noteEOFReader doesn't properly pick up
the EOF when the underlying io.Reader is a gzip.Reader, which in turn makes
http.persistConn think the subsequent Close() is an early close, which makes it not
reuse the connection for subsequent requests. I stumbled upon the issue when using
json.NewDecoder(resp.Body).Decode(...) and noticing connections not being reused. As a
workaround, I used ioutil.ReadAll() and json.Unmarshal(), and then boiled down the issue
to the attached files, with some advice and pointing in the right direction from Dave
Cheney on Twitter.

Attachments:

  1. server.go (709 bytes)
  2. client.go (623 bytes)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions