-
Notifications
You must be signed in to change notification settings - Fork 71
Closed
Description
We had a situation when the traffic got so high it was too much for the proxy instances we were running in our Kubernetes cluster - the default 1000 concurrent connections handled by fasthttp server was not enough. That situation revealed a bug in the proxy code, which resulted in pods restarting due to a SIGSEGV:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x976a2e]
goroutine 836 [running]:
main.(*processor).handle(0xc000170300, 0xc000219800)
/build/processor.go:191 +0x42e
github.com/valyala/fasthttp.(*Server).serveConn(0xc00015c400, {0xb62818?, 0xc0000ac6f0})
/go/pkg/mod/github.com/valyala/fasthttp@v1.51.0/server.go:2359 +0x120d
github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0000f4000, 0xc0001a9a40)
/go/pkg/mod/github.com/valyala/fasthttp@v1.51.0/workerpool.go:224 +0xa9
github.com/valyala/fasthttp.(*workerPool).getCh.func1()
/go/pkg/mod/github.com/valyala/fasthttp@v1.51.0/workerpool.go:196 +0x38
created by github.com/valyala/fasthttp.(*workerPool).getCh
/go/pkg/mod/github.com/valyala/fasthttp@v1.51.0/workerpool.go:195 +0x1b0
This seems to be caused by this code:
if r.err != nil {
ctx.Error(err.Error(), fh.StatusInternalServerError)
p.Errorf("src=%s req_id=%s: unable to proxy metadata: %s", clientIP, reqID, r.err)
return
}
in processor.go file. It checks if r.err is not nil, but in the next line tries to dereference err.Error().
Metadata
Metadata
Assignees
Labels
No labels