net/url: "host:port" URLs interpreted as "scheme:opaque"

[karalabe]

What version of Go are you using (go version)?

go version go1.8 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/work:/opt/google/cloud-sdk/platform/google_appengine/goroot/src"
GORACE=""
GOROOT="/opt/google/go"
GOTOOLDIR="/opt/google/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build015435601=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

https://play.golang.org/p/qbW1wWkcKY

What did you expect to see?

I would expect both URLs in the example code to be parsed the same way, and more importantly that the scheme for both is empty (and inherently IsAbs returning false for both).

What did you see instead?

The hostname in the second url was interpreted as the scheme and the port number as opaque. This causes IsAbs to return true. This makes IsAbs unsuitable to actually figure out if there's a scheme specified or not in some user input.

Although it's debatable whether example.com from example.com:80 can be interpreted as a scheme, I think it's not the intuitive choice. Alternatively I'd appreciate a way to check for "conventional" scheme's being present without having to so a strings.Contains("://") manually.

[mattn]

related #19297

[karalabe]

@mattn As I've mentioned, it may be the theoretically correct behavior according to the RFC, but I'm not sure it's the actual behavior developers expect. As such, many people might actually shoot themselves in the foot, just like we did.

[bradfitz]

Sorry, there's nothing more we can do here and stay within the go1compat promise.

[Hunsin]

I faced similar problem, too:
playground

I wonder if it's possible to return error if there's no scheme prefix and can't parse url.Host? Otherwise, developer need to validate raw string them self.

[mattn]

See rfc3986 section 3.1.

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

SCHEME can contain dot. So example.com://example.com is valid URL.

https://play.golang.org/p/sRUFtNKsN-

As I mentioned in #19297, please prepend :// into your hostname.

[gopherbot]

Change https://golang.org/cl/81436 mentions this issue: net/url: document Parse more