Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http/httputil: DumpRequestOut leaks goroutine when URL is invalid #32571

Closed
kristofferingemansson opened this issue Jun 12, 2019 · 4 comments
Assignees
Labels
Milestone

Comments

@kristofferingemansson
Copy link

@kristofferingemansson kristofferingemansson commented Jun 12, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.6 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\x\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=c:\Users\x\Documents\Go
set GOPROXY=
set GORACE=
set GOROOT=C:\Program Files\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\x\AppData\Local\Temp\go-build036886350=/tmp/go-build -gno-record-gcc-switches

What did you do?

When dumping an outgoing HTTP request with httputil.DumpRequestOut, a goroutine is leaked for each call when the URL is empty or somewhat invalid.

Non-complete? list of URL:s triggering this bug:
"" (empty string)
"http://" (no host)

https://play.golang.org/p/SVU8lqvL0JL

package main

import (
	"fmt"
	"net/http"
	"net/http/httputil"
	"runtime"
)

func main() {
	fmt.Println("start", runtime.NumGoroutine())
	
	for i := 0; i < 100; i++ {
		req, err := http.NewRequest(http.MethodGet, "", nil)
		if err != nil {
			panic(err)
		}
		_, err = httputil.DumpRequestOut(req, true)
		if err != nil {
			fmt.Println(err)
		}
	}

	fmt.Println("end", runtime.NumGoroutine())
}

What did you expect to see?

start 1
end 1

What did you see instead?

start 1
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
end 101
Output from pprof:
goroutine 13669 [chan send, 1289 minutes]:
net/http/httputil.DumpRequestOut.func2(0xc00015c278, 0xc0001564a0)
    /home/travis/.gimme/versions/go1.12.linux.amd64/src/net/http/httputil/dump.go:123 +0x1b2
created by net/http/httputil.DumpRequestOut
    /home/travis/.gimme/versions/go1.12.linux.amd64/src/net/http/httputil/dump.go:115 +0x3af
@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Jun 12, 2019

You are ignoring the error from DumpRequestOut btw. Not that it makes a difference though. I updated your example to print the error.

@agnivade agnivade changed the title httputil.DumpRequestOut leaks goroutine when URL is invalid net/http/httputil: DumpRequestOut leaks goroutine when URL is invalid Jun 12, 2019
@agnivade agnivade added this to the Go1.14 milestone Jun 12, 2019
@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Jun 12, 2019

@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Jun 12, 2019

I have a fix. Will send a CL.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jun 13, 2019

Change https://golang.org/cl/182037 mentions this issue: net/http/httputil: fix goroutine leak for DumpRequestOut

@gopherbot gopherbot closed this in e7a4ab4 Aug 28, 2019
tomocy added a commit to tomocy/go that referenced this issue Sep 1, 2019
When an invalid URL was passed to DumpRequestOut, it would directly return
without gracefully shutting down the reader goroutine.

So we create a channel and signal the reader goroutine to exit
if an error occurs during roundtrip.

Fixes golang#32571

Change-Id: I8c2970f1601e599f3d1ebfed298faf5f5716fc2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/182037
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
t4n6a1ka added a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
When an invalid URL was passed to DumpRequestOut, it would directly return
without gracefully shutting down the reader goroutine.

So we create a channel and signal the reader goroutine to exit
if an error occurs during roundtrip.

Fixes golang#32571

Change-Id: I8c2970f1601e599f3d1ebfed298faf5f5716fc2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/182037
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.