Skip to content

net/http: using Transport.IdleConnTimeout may lead to goroutine leak in an edge case #25621

Closed
@aryszka

Description

@aryszka

What version of Go are you using (go version)?

1.10.2

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOOS="linux"
GOARCH="amd64"

What did you do?

I ran an HTTP short-polling client and the polling timeout value was equal to the IdleConnTimeout of the used http.Transport.

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"runtime"
	"time"
)

func main() {
	// IdleConnTimeout close to the polling timeout
	pollTO := 30 * time.Millisecond
	connTO := pollTO

	// avoid the leak by not setting IdleConnTimeout:
	// connTO = 0
	// or by using a value different from the poll timeout:
	// connTO = pollTO + time.Millisecond

	// start a trivial server, it can be in a separate process:
	go func() {
		if err := http.ListenAndServe(":8080", http.HandlerFunc(
			func(w http.ResponseWriter, _ *http.Request) {
				w.Write([]byte("Hello, world!"))
			},
		)); err != nil {
			log.Fatalln(err)
		}
	}()

	// create a client with the IdleConnTimeout configured:
	c := &http.Client{
		Transport: &http.Transport{
			IdleConnTimeout: connTO,
		},
	}

	// run periodic requests, count the requests, goroutines and the heap in use:
	var (
		m runtime.MemStats
		reqs int
	)

	for {
		func() {
			req, err := http.NewRequest("GET", "http://localhost:8080", nil)
			if err != nil {
				log.Fatalln(err)
			}

			rsp, err := c.Do(req)
			if err != nil {
				log.Fatalln(err)
			}

			defer rsp.Body.Close()
			if _, err := ioutil.ReadAll(rsp.Body); err != nil {
				log.Fatalln(err)
			}

			reqs++
			runtime.ReadMemStats(&m)
			fmt.Print("\r",
				time.Now().Format("15:04:05"),
				", requests: ", reqs,
				", goroutines: ", runtime.NumGoroutine(),
				", heap in use: ", m.HeapInuse,
			)

			time.Sleep(pollTO)
		}()
	}
}

What did you expect to see?

I expected the existing connection from the pool either reused or not, without other issues (even if having the same timeouts for the short-polling and the IdleConnTimeout is quite an edge case).

What did you see instead?

I saw a steadily growing number of goroutines and memory usage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions