net/url: ResolveReference does not respect ForceQuery

[TimothyGu]

What version of Go are you using (go version)?

$ go version
go1.16.3

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/timothy-gu/.cache/go-build"
GOENV="/home/timothy-gu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/timothy-gu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/timothy-gu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3732439639=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/VIk2U1Wflir

import "net/url"

u, _ := url.Parse("https://timothygu.me/?hello")
u, _ = u.Parse("?")
fmt.Println(u)

What did you expect to see?

https://timothygu.me/?

This is the behavior of Chrome, Firefox, Safari, Node.js (both legacy and modern parsers), curl, Rust's url crate, and Ruby's URI module.

What did you see instead?

https://timothygu.me/?hello

Here's what RFC 3986 said about this case:

if (R.path == "") then
   T.path = Base.path;
   if defined(R.query) then
      T.query = R.query;
   else
      T.query = Base.query;
   endif;
else ...

where R is the parsed reference and T is the resolved target. Currently, net/url uses RawQuery != "" as the equivalent for "defined(R.query)". However, this does not take into account the ForceQuery flag added in Go 1.7, which should also satisfy "defined(R.query)".

To be fair, Go's current behavior is consistent with Python. However, Python's urllib doesn't distinguish between /? and / (i.e., it doesn't have a ForceQuery flag), so there's no way it can get this right.

---

Discovered through my URL Tester tool: https://timothygu.me/urltester/

[gopherbot]

Change https://golang.org/cl/317930 mentions this issue: net/url: consider ForceQuery in ResolveReference

[seankhliao]

cc @rsc @bradfitz