Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

Open
ggicci opened this issue Feb 11, 2019 · 8 comments
Open

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

ggicci opened this issue Feb 11, 2019 · 8 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Backlog

Comments

@ggicci
Copy link
Contributor

ggicci commented Feb 11, 2019 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

YES

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ggicci/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ggicci/workspace"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ht/hdwl47w16m10r45w89drkkph0000gn/T/go-build233590303=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I did use http.StripPrefix accompanied with http.ServeFile to strip URL prefix and serve a folder. And the HTTP server will panic and the web page will receive nothing if I opened http://localhost:8080/download/.

Here's a code snippet to reproduce the problem:

package main

import "net/http"

var myHandler = http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	http.ServeFile(rw, r, "./files")
})

func main() {
	http.Handle("/download/", http.StripPrefix("/download/", myHandler))
	http.ListenAndServe(":8080", nil)
}

What did you expect to see?

I should see contents from my files folder. File list or index page content.

What did you see instead?

No HTTP response, but panics information from stderr.

2019/02/11 17:43:17 http: panic serving [::1]:56154: runtime error: index out of range
goroutine 19 [running]:
net/http.(*conn).serve.func1(0xc00008a960)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12fdce0, 0xc000010030, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a000, 0xc000122000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a000, 0xc000122000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008a960, 0x1300100, 0xc000096200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56174: runtime error: index out of range
goroutine 20 [running]:
net/http.(*conn).serve.func1(0xc00008aa00)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12fdce0, 0xc00015a010, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e000, 0xc00016c000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e000, 0xc00016c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aa00, 0x1300100, 0xc000096300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56175: runtime error: index out of range
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc00008aaa0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12fdce0, 0xc000088d50, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc000194000, 0xc00010c300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc000194000, 0xc00010c300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aaa0, 0x1300100, 0xc000096480)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56176: runtime error: index out of range
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc0001ae000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12fdce0, 0xc000010080, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a0e0, 0xc000122300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a0e0, 0xc000122300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001ae000, 0x1300100, 0xc00005e200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:44:47 http: panic serving [::1]:56544: runtime error: index out of range
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc0001c2000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12fdce0, 0xc00015a070, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001c2000, 0x1300100, 0xc0001580c0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
@gopherbot
Copy link

Change https://golang.org/cl/161738 mentions this issue: net/http: StipPrefix clean URL path

@agnivade agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 11, 2019
@twmb twmb mentioned this issue Apr 23, 2019
Closed
@gopherbot
Copy link

Change https://golang.org/cl/180498 mentions this issue: net/http: roll back "clean the path of the stripped URL by StripPrefix"

@dmitshur
Copy link
Contributor

dmitshur commented Jun 4, 2019

I investigated this and here's some more information about why this happens.

The URL path for a GET /download/ request currently ends up being the empty string after http.StripPrefix("/download/", myHandler) strips the prefix.

Calling http.ServeFile on a *http.Request with the empty URL path panics in http.serveFile here:

// redirect if the directory name doesn't end in a slash
if d.IsDir() {
	url := r.URL.Path
	if url[len(url)-1] != '/' { // panic here, since url is empty string
		localRedirect(w, r, path.Base(url)+"/")
		return
	}
}

/cc @bradfitz

@gopherbot
Copy link

Change https://golang.org/cl/180499 mentions this issue: net/http: don't panic serving dir in ServeFile with empty Request.URL.Path

gopherbot pushed a commit that referenced this issue Jun 4, 2019
@dmitshur
net/http: roll back "clean the path of the stripped URL by StripPrefix"
003dbc4 
Roll back CL 161738. That fix changed StripPrefix behavior in the
general case, not just in the situation where where stripping the
prefix from path resulted in the empty string, causing issue #31622.

That kind of change to StripPrefix behavior is not backwards compatible,
and there can be a smaller, more targeted fix for the original issue.

Fixes #31622
Updates #30165

Change-Id: Ie2fcfe6787a32e44f71d564d8f9c9d580fc6f704
Reviewed-on: https://go-review.googlesource.com/c/go/+/180498
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur
Copy link
Contributor

dmitshur commented Jun 4, 2019

Re-opening this because the fix was rolled back in CL 180498.

For (my) reference, the code where panic happens was added in CL 20128 to fix #13996. It didn't take into account that r.URL.Path can be empty string when http.StripPrefix is used in a way that over-strips the prefix.

When fixing this, need to make sure it won't introduce an infinite or otherwise unwanted redirects. The analysis in the comment #13996 (comment) is good and should be taken into account.

@dmitshur dmitshur reopened this Jun 4, 2019
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 4, 2019
@dmitshur dmitshur self-assigned this Jun 4, 2019
@dmitshur dmitshur added this to the Go1.13 milestone Jun 4, 2019
@dmitshur dmitshur changed the title net/http: StripPrefix makes the stripped URL having an empty root path rather than "/" net/http: ServeFile panics when StripPrefix over-strips and results in empty path Jun 5, 2019
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@odeke-em
Copy link
Member

I see this issue was moved to Go1.14 but since it is a regression perhaps for Go1.13, but also Brad's CL https://golang.org/cl/180499 LGTM, just that I don't have access to update his CL with a code review comment that was requested to use strings.HasPrefix instead of a plain comparison to ""

@dmitshur @andybons can we perhaps make that happen and submit that CL? Thank you.

@andybons
Copy link
Member

@odeke-em it’s too late for this to go into 1.13 at this point. Sorry.

@odeke-em
Copy link
Member

odeke-em commented Aug 2, 2019

Got it! We can wait until Go.1.14 then :)

gopherbot pushed a commit that referenced this issue Aug 28, 2019
@bradfitz
net/http: don't panic serving dir in ServeFile with empty Request.URL…
7ed973b 
….Path

Updates #30165
Updates #31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
tomocy pushed a commit to tomocy/go that referenced this issue Sep 1, 2019
@bradfitz @tomocy
net/http: don't panic serving dir in ServeFile with empty Request.URL…
f9f34ba 
….Path

Updates golang#30165
Updates golang#31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
t4n6a1ka pushed a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
@bradfitz @t4n6a1ka
net/http: don't panic serving dir in ServeFile with empty Request.URL…
9ca062b 
….Path

Updates golang#30165
Updates golang#31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
7 participants
@rsc @andybons @agnivade @dmitshur @ggicci @odeke-em @gopherbot