net/url: export escaper with mode encodePath

[movsb]

What version of Go are you using (go version)?

$ go version
go version go1.13.4 darwin/amd64

Does this issue reproduce with the latest release?

It does.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tao.yang/Library/Caches/go-build"
GOENV="/Users/tao.yang/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tao.yang/codes/gocodes"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/tao.yang/codes/gocodes/src/github.com/elastic/beats/filebeat/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2l/zkndzzg90t746hcwzwb9pbz00000gn/T/go-build399229267=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Use url.PathEscape to escape file system path.

package main

import (
	"fmt"
	"net/url"
)

func main() {
	fmt.Println(url.PathEscape((`/a?/b`)))
}

Notice that ? is a valid path character on Unix-like systems.
Don't we treat it as a query that follows.

What did you expect to see?

/a%3F/b

What did you see instead?

%2Fa%3F%2Fb

Why?

PathEscape is actually the path segment escaper.

// PathEscape escapes the string so it can be safely placed inside a URL path segment,
// replacing special characters (including /) with %XX sequences as needed.
func PathEscape(s string) string {
	return escape(s, encodePathSegment)
}

What I need is a path escaper, with mode == encodePath, not encodePathSegment. Something like:

func RealPathEscape(s string) string {
	return escape(s, encodePath)
}

Workarounds

escape path segments individually

package main

import (
	"fmt"
	"net/url"
	"strings"
)

func main() {
	segments := strings.Split(`/a?/b`, "/")
	for i, segment := range segments {
		segments[i] = url.PathEscape(segment)
	}
	fmt.Println(strings.Join(segments, "/"))
}

use EscapedPath

package main

import (
	"fmt"
	"net/url"
)

func main() {
	u := url.URL{Path: "/a?/b"}
	fmt.Println(u.EscapedPath())
}

[toothrot]

I'm not sure I understand. If ? is a valid character in a filesystem path name, /a%3F/b would be incorrect. What are you trying to escape, or how are you using this escaped value?

[movsb]

Because the RealPathEscape does what http.FileServer does:

go/src/net/http/fs.go

Lines 119 to 123 in 7d30af8

	// name may contain '?' or '#', which must be escaped to remain
	// part of the URL path, and not indicate the start of a query
	// string or fragment.
	url := url.URL{Path: name}
	fmt.Fprintf(w, "<a href=\"%s\">%s</a>\n", url.String(), htmlReplacer.Replace(name))

Here, url.String() == url.EscapedPath().

For example, if we have a static file server:

package main

import "net/http"

func main() {
    http.Handle("/", http.FileServer(http.Dir(".")))
    http.ListenAndServe(":1234", nil)
}

Let's cURL it:

$ curl localhost:1234
<pre>
<a href="a%3F/">a?/</a>
<a href="server.go">server.go</a>
</pre>

Note that a?/ is escaped to a%3F/.

---

how are you using this escaped value?

While I'm manually rendering an HTML page.

[toothrot]

/cc @bradfitz

[bradfitz]

The URL.EscapedPath method already does what you want:

https://play.golang.org/p/cOFqKqyM88y

fmt.Println((&url.URL{Path: "/This is/a path/?foo=bar"}).EscapedPath())

/This%20is/a%20path/%3Ffoo=bar

(The implementation of EscapedPath is basically just return escape(u.Path, encodePath))

Given what a fringe use case this is, I don't think we want to add a top-level func for this.

[movsb]

@bradfitz So, you are saying that PathEscape() ≠ EscapedPath(). That's really confusing.