net/http: Server.Shutdown waits connections which are in new state, but Client puts such connections into idle connections pool

[pamburus]

It looks like there is odd logic both in http.Client and http.Server which in couple trigger really undesired behaviour.

1. http.Server.Shutdown does not exit until there are any connections not in IDLE state, including connections in NEW state. NEW connections really do not significantly differ from IDLE. Both of them are not processing any requests, either already, or yet.
2. http.Client creates new connection in case when there are no more IDLE connections in the pool, but when connection is established, it realises that other IDLE connection already appeared in the pool, and it prefers existing IDLE connection over newly created one. Just created connection is put to the idle connections pool, but there were no any requests sent over it. http.Server treats such connections as NEW and waits while client sends a request before shutting down.

What version of Go are you using (go version)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/media/psf/Home/Documents/Work/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build834255989=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

[Example 1] https://play.golang.org/p/l22PCUvqpf

1. Create and start http.Server
2. Make a TCP connection to it
3. Do no send any data to the connection
4. Shutdown server

[Example 2] https://play.golang.org/p/JrZPNM3cn7

1. Create http.Client
2. Create http.Server
3. Send a bunch of requests to server via http.Client
4. Stop the server
5. Go to 2

What did you expect to see?

1. Example 1: http.Server.Shutdown do not wait for connections which are not active yet. NEW connections are logically the same as IDLE connections, there is no need to wait for them.
2. Example 1: HTTP server with default parameters has reasonable timeouts which are suitable for most non-advanced cases.
3. Example 2: HTTP client sends the request which triggered creation of a new connection, over that just created connection. There is no any reason to suddenly switch to another connection.

What did you see instead?

1. Example 1: HTTP server closes only IDLE connections and waits until all connections become IDLE or closed.
2. Example 1: HTTP server with default parameters has infinite timeouts which makes it practically unusable.
3. Example 2: HTTP client puts just created connection to IDLE connections pool without sending requests to it.

[bradfitz]

/cc @tombergan

[tombergan]

I'm going to close this as a dup of #22682. Please comment there if we missed anything, or reopen this issue if you think it's not a dup.

@tombergan
I'm not sure anyone but contributors can reopen (their own) issues here. I can't.

[ianlancetaylor]

@opennota If you think this should be reopened, let us know, and we will reopen it for you.