net/http: HTTP requests with InsecureSkipVerify transport don't close TCP connections

[knadh]

1. What version of Go are you using (go version)?
   go version go1.6.2 linux/amd64
2. What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/code/go"
GORACE=""
GOROOT="/usr/lib/go-1.6"
GOTOOLDIR="/usr/lib/go-1.6/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

1. What did you do?

Run the following program. It creates an HTTP server on :8000 and spawns a goroutine that sends GET requests to the server every 1 second. The server writes nothing in the response body; body-length = 0. The client makes GET requests and does a body.Close() without reading the body. Note that the transport has InsecureSkipVerify.

package main

import (
    "crypto/tls"
    "fmt"
    "log"
    "net/http"
    "time"
)

func Server(w http.ResponseWriter, req *http.Request) {
    // If a response is written, the issue isn't triggered.
    // io.WriteString(w, "hello, world!\n")
}

func Client(url string) {
    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        log.Fatal(err)
    }

    // This transport is what's causing unclosed connections.
    tr := &http.Transport{
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    }
    hc := &http.Client{Timeout: 2 * time.Second, Transport: tr}

    resp, err := hc.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
}

func main() {
    // Wait a bit and launch one client every second.
    // On the terminal, run `ss | grep :8000` to see
    // ESTABLISHED connections grow with every request.

    go func() {
        time.Sleep(1 * time.Second)

        n := 0
        for {
            Client("http://localhost:8000")
            fmt.Println("Request", n)
            n += 1

            time.Sleep(1 * time.Second)
        }
    }()

    http.HandleFunc("/", Server)
    log.Fatal(http.ListenAndServe(":8000", nil))
}

1. What did you expect to see?
   Every request the client makes should get a 200 response from the server and immediately close the connection.
2. What did you see instead?
   The client keeps making requests and gets 200 back, but the underlying TCP connection is never closed. It's stuck in the ESTABLISHED state perpetually.

Check the open connections by running ss | grep :8000 every few seconds. They keep growing. netstat n | grep :8000 also shows TIME_WAIT connections.

• Writing a response in Server() solves the issue.
• Removing InsecureSkipVerify solves the issue.

[bradfitz]

This has nothing to do with InsecureSkipVerify.

You're leaking Transports which are holding TCP connections.

See the documentation under https://golang.org/pkg/net/http/#Transport :

By default, Transport caches connections for future re-use. This may leave many open connections when accessing many hosts. This behavior can be managed using Transport's CloseIdleConnections method and the MaxIdleConnsPerHost and DisableKeepAlives fields.

Transports should be reused instead of created as needed. Transports are safe for concurrent use by multiple goroutines.

Don't create a Transport for every request.