Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: populate Uncompressed Content-Length #38928

Open
FireFart opened this issue May 7, 2020 · 1 comment
Open

net/http: populate Uncompressed Content-Length #38928

FireFart opened this issue May 7, 2020 · 1 comment
Labels
Milestone

Comments

@FireFart
Copy link

@FireFart FireFart commented May 7, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/firefart/.cache/go-build"
GOENV="/home/firefart/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/firefart/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build918042854=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Currently when the net/http package receives a compressed response, the ContentLength is set to -1 and removed from the Header array:
https://golang.org/pkg/net/http/#Response (See field Uncompressed)

Often we are not interested in the body but only the Content-Length. If that's the case we often do io.Copy(ioutil.Discard, resp.Body) to consume the body.

Without checking the body length or the return from ioutil.Discard it's not possible to get the uncompressed Content-Length from the Response.

What did you expect to see?

I would suggest to introduce a new Property in the Response like UncompressedContentLength to make the uncompressed size available to the caller before the body is read. At this stage the size should already be known to go, as the Uncompressed property has already been processed.

What did you see instead?

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented May 7, 2020

Knowing that the body is compressed is different from knowing the size of the body.
The server may continue to stream the body, so you'd have no way of knowing the size of the body without reading it all.

@ALTree ALTree added the NeedsDecision label May 8, 2020
@ALTree ALTree changed the title net/http: Populate Uncompressed Content-Length net/http: populate Uncompressed Content-Length May 8, 2020
@ALTree ALTree added this to the Unplanned milestone May 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.