net/http: PUT: DefaultClient significantly slower than DefaultTransport when network is slow #26702
Comments
For a 30MB file we saw difference of about 3minutes+ with Go net/http when compared to Curl. |
Is this reproducible with client and server being on the same machine ? Would like to investigate this on my laptop. |
@agnivade Yes this is continuously reproducible, provided we delay the n/w. |
Oh, I see this is for http. Could you also kindly check for http2 client and see if you get similar results ? This would help us pin-point the issue faster. |
I tested with http2 transport and there is a small difference between Go code and Curl. |
@subbu05 have you had a chance to run the same code using http.DefaultTransport which is the core of the Client? That could help us narrow down where the bottleneck is. |
/cc @bradfitz |
@meirf I tested with http.DefaultTransport and I see curl and golang client file upload are all most ~same rate. |
@subbu05 using your exact code as the client and the below Go code as the server, I am not seeing any difference in performance vs curl. Can you please show us the exact command you are using to run https://github.com/sashgorokhov/docker-nginx-webdav? Go server:
|
@meirf
Observed: |
@meirf, the title of this bug no longer makes sense, since DefaultClient uses DefaultTransport. I don't see how they could have different speeds. |
What version of Go are you using (go version)?
go version go1.10.3 darwin/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN="/Users/test/go/bin"
GOCACHE="/Users/test/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/test/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.3/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/76/trlnwqwj0bs5003x850kzpg00000gn/T/go-build781190647=/tmp/go-build -gno-record-gcc-switches -fno-common"
Scenario:
I have a network with Download speed :400kbit/s Upload:400kbit/s RTT:200ms
I have http client developed in Golang.
Wanted to transfer 1MB file to a web server.
What did you expect to see?
File upload must almost be same to curl or other web clients.
What did you see instead?
Curl took ~28s
Go client took ~42s
More Information on environment, tools:
How does Go react when there are issues with network.
To mimic network delay - https://github.com/sitespeedio/throttle
web server : https://github.com/sashgorokhov/docker-nginx-webdav
Source Code
Tried with and without Content-Length header(chunked encoding) in the http request.
My concern is for 1mb file I see a difference what if file size is 100MB+?
The text was updated successfully, but these errors were encountered: