Repository navigation
net/http: DefaultServeMux incorrectly redirect (301) to path if path includes %2F%2F #21955
Description
Activity
- changed the title
[-]net/http incorrectly redirect (301) to path if path includes `%2F%2F`[/-][+]net/http.DefaultServeMux incorrectly redirect (301) to path if path includes `%2F%2F`[/+]on Sep 21, 2017 FWIW it happens even with
http://localhost:8878//http://localhost:8878/fdsfklasdjfldsajfldksajf/sdfssdf//sdfdsaf
The key aspect being two consecutive
/s. It's redirected to a path without two/s.Redirecting
http://localhost:8878//tohttp://localhost:8878/is correct.
But redirectinghttp://localhost:8878/%2Ftohttp://localhost:8878/is not correct.
I think this is a bug to be fixed.Reacted by José Francisco Calvo- changed the title
[-]net/http.DefaultServeMux incorrectly redirect (301) to path if path includes `%2F%2F`[/-][+]net/http: DefaultServeMux incorrectly redirect (301) to path if path includes `%2F%2F`[/+]on Mar 30, 2018 I fixed similar bug in the past, so I will confirm this.
Change https://golang.org/cl/103696 mentions this issue:
net/http: avoid incorrect redirection on paths containing encoded slashesI'm not sure if this actually a bug in the first place. The URL reported in the original comment is:
http://localhost:8878/path:a%2F%2Fgoogle.com%2FThe decoded path is:
/path:a//google.com/ServerMux.Handler uses path.Clean. That path clearly has a double slash and path.Clean translates double slashes to single slashes. Also see here. The godoc for ServerMux.Handler says:
// If the path is not in its canonical form, the // handler will be an internally-generated handler that redirects // to the canonical path.If there's an open question here, the question is around the definition of "canonical path". Does the canonical path convert double slashes to single slashes? If so, does it also convert %2F%2F to "/"? If yes to both questions, there's no bug. Otherwise, there's a bug. AFAICT, these questions are not answered in any documentation. In net/http/server_tests.go, serverMuxTests does not have a case for double slashes, so I suspect this question hasn't come up until now.
@tombergan Thank you for the clear explanation. I'll investigate this problem again.
As a countermeasure against this similar problem, there is nginx
merge_slashessyntax.
Even with nginx, the above options are used for cases where you do not want to integrate encoded duplicate slashes into one.And then, I have take a look at RFC-3986 for confirming the encoded double slashes validity.
According to it, the encoded string pchar is considered valid.path = path-abempty ; begins with "/" or is empty / path-absolute ; begins with "/" but not "//" / path-noscheme ; begins with a non-colon segment / path-rootless ; begins with a segment / path-empty ; zero characters path-abempty = *( "/" segment ) path-absolute = "/" [ segment-nz *( "/" segment ) ] path-noscheme = segment-nz-nc *( "/" segment ) path-rootless = segment-nz *( "/" segment ) path-empty = 0<pchar> segment = *pchar segment-nz = 1*pchar segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) ; non-zero-length segment without any colon ":" pchar = unreserved / pct-encoded / sub-delims / ":" / "@" query = *( pchar / "/" / "?" ) fragment = *( pchar / "/" / "?" ) pct-encoded = "%" HEXDIG HEXDIGTherefore, I think, if we follow RFC-3986, the current behavior is buggy.
Reacted by Wèi Cōngruì and José Francisco Calvo@bradfitz @tombergan ping?
- added a commit that references this issue
on May 20, 2020 Change https://golang.org/cl/234657 mentions this issue:
net/http: prevent incorrect redirections when the path contains %2F%2FTo expand on @tombergan's comment (#21955 (comment)), the
ServeMuxdocumentation states:ServeMux also takes care of sanitizing the URL request path and the Host header, stripping the port number and redirecting any request containing . or .. elements or repeated slashes to an equivalent, cleaner URL.
The documentation is not explicit on whether
%2fis considered a slash, but the implementation consistently treats it as such.ServeMux.Handle("/foo/", f)matches the request/foo%2fbar.I believe the current behavior is consistent with the documentation and probably can't be changed without violating compatibility. Perhaps the documentation should explicitly mention the treatment of escaped path components.
Reacted by Wèi CōngruìI'm suffering the same problem trying to use an encoded full URL in the path.
For example I'm trying to use this URL (an image from Wikimedia):
https://upload.wikimedia.org/wikipedia/commons/4/49/Mosuo_Woman_-_42723465920.jpgOnce encoded we get the following URL:
https%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F4%2F49%2FMosuo_Woman_-_42723465920.jpgAnd this is the request to my server using the previous URL as path:
http://localhost:8080/https%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F4%2F49%2FMosuo_Woman_-_42723465920.jpgI'm receiving a
301status code with aMoved Permanentlybody message linking to:http://localhost:8080/https:/upload.wikimedia.org/wikipedia/commons/4/49/Mosuo_Woman_-_42723465920.jpgI understand that
ServeMuxis cleaning paths and redirecting in the case of differences but what I don't understand it's why that is done for escaped characters like%2F%2F.Apart from the possible bug, anybody knows any solution/workaround that doesn't involve creating from scratch a custom serve mux or using another one available? Any possible way to avoid that cleaning?
This is clearly a bug.
The encoding method can encode arbitrary data in a Uniform Resource Identifier (URI), which is the point of the encoding.
"/" is the reserved character slash. "%2f" is an arbitrary string.
"/" and "%2f" are two different things.Is "%2f" is a slash?
No, of course it is not a slash. and "%25" is not a percent sign
Otherwise "https://www.google.com/" will be equivalent to "https:%252f%252525252fwww.google.com%2f".Reacted by José Francisco Calvo, Kobe Lipkens, pýrus and ALX99Just faced this issue in the latest go 1.18.3
curl http://localhost/%2Fhellois decoded internally to//hello(r.URL.Path) and the decision for a 301 redirect based onr.URL.Pathinstead ofr.URL.RawPath.A dirty hack that fixed useless 301 redirect issue for me:
--- go/src/net/http/server.go 2022-06-01 18:44:51.000000000 +0200 +++ go_/src/net/http/server.go 2022-06-15 23:44:02.602151436 +0200 @@ -2412,7 +2412,7 @@ // All other requests have any port stripped and path cleaned // before passing to mux.handler. host := stripHostPort(r.Host) - path := cleanPath(r.URL.Path) + path := cleanPath(r.URL.RawPath) // If the given path is /tree and its handler is not registered, // redirect for /tree/. @@ -2420,7 +2420,7 @@ return RedirectHandler(u.String(), StatusMovedPermanently), u.Path } - if path != r.URL.Path { + if path != r.URL.RawPath { _, pattern = mux.handler(host, path) u := &url.URL{Path: path, RawQuery: r.URL.RawQuery} return RedirectHandler(u.String(), StatusMovedPermanently), pattern
Reacted by Tommy D. RossiChange https://go.dev/cl/528356 mentions this issue:
net/http: ServeMux matches escaped pathsChange https://go.dev/cl/530575 mentions this issue:
net/http: unescape paths and patterns by segment- locked and limited conversation to collaborators
on Sep 24, 2024
What version of Go are you using (
go version)?go version go1.9 linux/amd64Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env)?What did you do?
What did you expect to see?
404 page not foundWhat did you see instead?
<a href="/path:a/google.com/">Moved Permanently</a>.NOTE: this does not happen if alice was used instead of the
http.DefaultServeMux