-
Notifications
You must be signed in to change notification settings - Fork 18.3k
Description
What version of Go are you using (go version
)?
$ go version go version go1.17.5 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env
)?
go env
Output
$ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GOVCS="" GOVERSION="go1.17.5" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0h/wc_q919d4_9_f0wj1p4v4h840000gn/T/go-build4221745526=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
I'm seeing the first http request using net/http
package and Client
to be consistently 100-200ms slower than the consecutive ones. This happens both when sending requests to one or multiple servers. The code to reproduce the issue:
package main
import (
"crypto/tls"
"fmt"
"net/http"
"net/http/httptrace"
"time"
)
func main() {
request := func (client *http.Client, url string) {
t := time.Now()
fmt.Println("")
clientTrace := &httptrace.ClientTrace{
GetConn: func(hostPort string) { s := time.Now(); fmt.Println(s.Sub(t), "starting to create conn ", hostPort); t = s },
DNSStart: func(info httptrace.DNSStartInfo) { s := time.Now(); fmt.Println(s.Sub(t), "starting to look up dns", info); t = s },
DNSDone: func(info httptrace.DNSDoneInfo) { s := time.Now(); fmt.Println(s.Sub(t), "done looking up dns", info); t = s },
ConnectStart: func(network, addr string) { s := time.Now(); fmt.Println(s.Sub(t), "starting tcp connection", network, addr); t = s },
ConnectDone: func(network, addr string, err error) { s := time.Now(); fmt.Println(s.Sub(t), "tcp connection created", network, addr, err) ; t = s},
GotConn: func(info httptrace.GotConnInfo) { s := time.Now(); fmt.Println(s.Sub(t), "connection established", info.Reused); t = s },
GotFirstResponseByte: func() { s := time.Now(); fmt.Println(s.Sub(t), "got first byte"); t = s },
TLSHandshakeStart: func() { s := time.Now(); fmt.Println(s.Sub(t), "handshake start"); t = s },
TLSHandshakeDone: func(tt tls.ConnectionState, e error) {s := time.Now(); fmt.Println(s.Sub(t), "handshake end"); t = s},
}
req, _ := http.NewRequest("GET", url, nil)
req.Close = true
clientTraceCtx := httptrace.WithClientTrace(req.Context(), clientTrace)
req = req.WithContext(clientTraceCtx)
response, _ := client.Do(req)
response.Body.Close()
}
client := &http.Client{}
request(client, "https://news.ycombinator.com")
client = &http.Client{}
request(client, "https://news.ycombinator.com")
client = &http.Client{}
request(client, "https://news.ycombinator.com")
}
What did you expect to see?
Roughly the same timings for all the requests.
What did you see instead?
145.166µs starting to create conn news.ycombinator.com:443
276.459µs starting to look up dns {news.ycombinator.com}
58.451916ms done looking up dns {[{209.216.230.240 }] <nil> false}
616.542µs starting tcp connection tcp 209.216.230.240:443
190.6365ms tcp connection created tcp 209.216.230.240:443 <nil>
711.875µs handshake start
517.936208ms handshake end
620.167µs connection established false
194.723083ms got first byte
115.5µs starting to create conn news.ycombinator.com:443
93.959µs starting to look up dns {news.ycombinator.com}
8.060583ms done looking up dns {[{209.216.230.240 }] <nil> false}
41.167µs starting tcp connection tcp 209.216.230.240:443
179.021541ms tcp connection created tcp 209.216.230.240:443 <nil>
224.875µs handshake start
362.071834ms handshake end
152.291µs connection established false
180.119625ms got first byte
91.333µs starting to create conn news.ycombinator.com:443
129.208µs starting to look up dns {news.ycombinator.com}
2.597417ms done looking up dns {[{209.216.230.240 }] <nil> false}
74.833µs starting tcp connection tcp 209.216.230.240:443
181.5515ms tcp connection created tcp 209.216.230.240:443 <nil>
151.334µs handshake start
359.525791ms handshake end
157.042µs connection established false
179.980583ms got first byte
The slowdown for the first request happens for every server I tried and it's always only the first request, whether the Client is reused or recreated for every request.
Just to make sure that there's no confusion - I tried this for multiple servers, the first request is always slower. For server X, the average request duration is N if it's the first request and M if it's one of the consecutive requests. Duration M for consecutive requests is roughly the same across multiple runs regardless of whether the first request was to the server X or some other server. The duration M is always 100-200ms lower than N.
The logs I posted seem to indicate that the bulk of the slowdown is coming from TLS handshake. I couldn't find anything in the code that would point to some overhead/initialization happening exclusively for the first request.
I'm not necessarily saying there's a bug - this might be the expected behavior. In that case it would be great to update the documentation, I spend few hours researching this and found no explanation for this behavior.
Thank you for your time!