Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

Open
ggicci opened this issue Feb 11, 2019 · 8 comments
Open
Assignees
Labels
Milestone

Comments

@ggicci
Copy link
Contributor

@ggicci ggicci commented Feb 11, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

YES

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ggicci/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ggicci/workspace"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ht/hdwl47w16m10r45w89drkkph0000gn/T/go-build233590303=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I did use http.StripPrefix accompanied with http.ServeFile to strip URL prefix and serve a folder. And the HTTP server will panic and the web page will receive nothing if I opened http://localhost:8080/download/.

Here's a code snippet to reproduce the problem:

package main

import "net/http"

var myHandler = http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	http.ServeFile(rw, r, "./files")
})

func main() {
	http.Handle("/download/", http.StripPrefix("/download/", myHandler))
	http.ListenAndServe(":8080", nil)
}

What did you expect to see?

I should see contents from my files folder. File list or index page content.

What did you see instead?

No HTTP response, but panics information from stderr.

2019/02/11 17:43:17 http: panic serving [::1]:56154: runtime error: index out of range
goroutine 19 [running]:
net/http.(*conn).serve.func1(0xc00008a960)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12fdce0, 0xc000010030, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a000, 0xc000122000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a000, 0xc000122000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008a960, 0x1300100, 0xc000096200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56174: runtime error: index out of range
goroutine 20 [running]:
net/http.(*conn).serve.func1(0xc00008aa00)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12fdce0, 0xc00015a010, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e000, 0xc00016c000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e000, 0xc00016c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aa00, 0x1300100, 0xc000096300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56175: runtime error: index out of range
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc00008aaa0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12fdce0, 0xc000088d50, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc000194000, 0xc00010c300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc000194000, 0xc00010c300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aaa0, 0x1300100, 0xc000096480)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56176: runtime error: index out of range
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc0001ae000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12fdce0, 0xc000010080, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a0e0, 0xc000122300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a0e0, 0xc000122300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001ae000, 0x1300100, 0xc00005e200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:44:47 http: panic serving [::1]:56544: runtime error: index out of range
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc0001c2000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12fdce0, 0xc00015a070, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001c2000, 0x1300100, 0xc0001580c0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 11, 2019

Change https://golang.org/cl/161738 mentions this issue: net/http: StipPrefix clean URL path

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jun 4, 2019

Change https://golang.org/cl/180498 mentions this issue: net/http: roll back "clean the path of the stripped URL by StripPrefix"

@dmitshur

This comment has been minimized.

Copy link
Member

@dmitshur dmitshur commented Jun 4, 2019

I investigated this and here's some more information about why this happens.

The URL path for a GET /download/ request currently ends up being the empty string after http.StripPrefix("/download/", myHandler) strips the prefix.

Calling http.ServeFile on a *http.Request with the empty URL path panics in http.serveFile here:

// redirect if the directory name doesn't end in a slash
if d.IsDir() {
	url := r.URL.Path
	if url[len(url)-1] != '/' { // panic here, since url is empty string
		localRedirect(w, r, path.Base(url)+"/")
		return
	}
}

/cc @bradfitz

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jun 4, 2019

Change https://golang.org/cl/180499 mentions this issue: net/http: don't panic serving dir in ServeFile with empty Request.URL.Path

gopherbot pushed a commit that referenced this issue Jun 4, 2019
Roll back CL 161738. That fix changed StripPrefix behavior in the
general case, not just in the situation where where stripping the
prefix from path resulted in the empty string, causing issue #31622.

That kind of change to StripPrefix behavior is not backwards compatible,
and there can be a smaller, more targeted fix for the original issue.

Fixes #31622
Updates #30165

Change-Id: Ie2fcfe6787a32e44f71d564d8f9c9d580fc6f704
Reviewed-on: https://go-review.googlesource.com/c/go/+/180498
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur

This comment has been minimized.

Copy link
Member

@dmitshur dmitshur commented Jun 4, 2019

Re-opening this because the fix was rolled back in CL 180498.

For (my) reference, the code where panic happens was added in CL 20128 to fix #13996. It didn't take into account that r.URL.Path can be empty string when http.StripPrefix is used in a way that over-strips the prefix.

When fixing this, need to make sure it won't introduce an infinite or otherwise unwanted redirects. The analysis in the comment #13996 (comment) is good and should be taken into account.

@dmitshur dmitshur reopened this Jun 4, 2019
@dmitshur dmitshur added NeedsFix and removed NeedsInvestigation labels Jun 4, 2019
@dmitshur dmitshur self-assigned this Jun 4, 2019
@dmitshur dmitshur added this to the Go1.13 milestone Jun 4, 2019
@dmitshur dmitshur changed the title net/http: StripPrefix makes the stripped URL having an empty root path rather than "/" net/http: ServeFile panics when StripPrefix over-strips and results in empty path Jun 5, 2019
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Jul 31, 2019

I see this issue was moved to Go1.14 but since it is a regression perhaps for Go1.13, but also Brad's CL https://golang.org/cl/180499 LGTM, just that I don't have access to update his CL with a code review comment that was requested to use strings.HasPrefix instead of a plain comparison to ""

@dmitshur @andybons can we perhaps make that happen and submit that CL? Thank you.

@andybons

This comment has been minimized.

Copy link
Member

@andybons andybons commented Jul 31, 2019

@odeke-em it’s too late for this to go into 1.13 at this point. Sorry.

@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Aug 2, 2019

Got it! We can wait until Go.1.14 then :)

gopherbot pushed a commit that referenced this issue Aug 28, 2019
….Path

Updates #30165
Updates #31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
tomocy added a commit to tomocy/go that referenced this issue Sep 1, 2019
….Path

Updates golang#30165
Updates golang#31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
t4n6a1ka added a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
….Path

Updates golang#30165
Updates golang#31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.