New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/url: fails to decode %ya whereas browsers are more tolerant #29808

Open
Darkemon opened this Issue Jan 18, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@Darkemon
Copy link

Darkemon commented Jan 18, 2019

What version of Go are you using (go version)?

go version go1.11.4 freebsd/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="freebsd" GOOS="freebsd" GOPATH="/root/go" GOPROXY="" GORACE="" GOROOT="/usr/local/go" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64" GCCGO="gccgo" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build246043989=/tmp/go-build -gno-record-gcc-switches"
$ go env

What did you do?

Yandex web application (https://yandex.ru/search) periodically sends requests like:

https://yandex.ru/clck/click/reqid=1545391593487252-912524167688176914537851-man1-1492/path=690.491.59/vars=-no=19,-blob=aYLIB2m%yAdp%sgHabbJBw__/*https://yandex.ru/search/?text=%D0%BF%D1%80%D0%BE%D0%BC%D0%BE%D0%BA%D0%BE%D0%B4%20%D0%BE%D1%81%D0%B5%D1%82%D0%B8%D0%BD%D1%81%D0%BA%D0%B8%D0%B5%20%D0%BF%D0%B8%D1%80%D0%BE%D0%B3%D0%B8%20ospirogi&lr=213

I parse this URL with url.ParseRequestURI() and it returns an error, but as I understand the URL is valid.

What did you expect to see?

Parsed URL.

What did you see instead?

The error:

 parse https://yandex.ru/clck/click/reqid=1545391593487252-912524167688176914537851-man1-1492/path=690.491.59/vars=-no=19,-blob=aYLIB2m%yAdp%sgHabbJBw__/*https://yandex.ru/search/?text=%D0%BF%D1%80%D0%BE%D0%BC%D0%BE%D0%BA%D0%BE%D0%B4%20%D0%BE%D1%81%D0%B5%D1%82%D0%B8%D0%BD%D1%81%D0%BA%D0%B8%D0%B5%20%D0%BF%D0%B8%D1%80%D0%BE%D0%B3%D0%B8%20ospirogi&lr=213: invalid URL escape "%yA"
@aslrousta

This comment has been minimized.

Copy link

aslrousta commented Jan 18, 2019

According to the standard RFC 3986, Section 2.1, a percent encoded character must be of the form:

pct-encoded = "%" HEXDIG HEXDIG

And, the percent sign (%) itself must be encoded as:

Because the percent ("%") character serves as the indicator for percent-encoded octets, it must be percent-encoded as "%25" for that octet to be used as data within a URI.

So the character sequence %yA is an invalid percent encoded character for sure. Although, most URL parsers (and especially Web browsers) are more tolerant against such errors.

@bradfitz bradfitz added this to the Go1.13 milestone Jan 18, 2019

@bradfitz bradfitz changed the title net/url: incorrectly unescapes path in URL net/url: fails to decode %ya whereas browsers are more tolerant Jan 18, 2019

@Darkemon

This comment has been minimized.

Copy link
Author

Darkemon commented Jan 19, 2019

Thanks, I'll be waiting for decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment