Skip to content

net/http: use nontransitional IDNA processing #46001

Closed
@TimothyGu

Description

@TimothyGu

What version of Go are you using (go version)?

$ go version
go version go1.16.3 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/timothy-gu/.cache/go-build"
GOENV="/home/timothy-gu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/timothy-gu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/timothy-gu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/gourl/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build343371698=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
	"fmt"
	"net/http"
)

func main() {
	resp, _ := http.Get("http://straße.de/")
	fmt.Println("straße", resp.Header.Get("Content-Length"))

	resp, _ = http.Get("http://strasse.de/")
	fmt.Println("strasse", resp.Header.Get("Content-Length"))

	resp, _ = http.Get("http://xn--strae-oqa.de/")
	fmt.Println("xn--strae-oqa", resp.Header.Get("Content-Length"))
}

What did you expect to see?

straße.de should get resolved to xn--strae-oqa per IDNA Nontransitional processing and DENIC announcement. This is the case on Firefox and Safari when you type the URL into the address bar. It is also the behavior of curl, wget, Node.js, and the browser WHATWG URL Standard.

straße 33560
strasse 6637
xn--strae-oqa 33560

What did you see instead?

straße.de got resolved instead to strasse.de. This is the current behavior in Chrome, and from what Ryan Sleevi implied in https://crbug.com/694157, other pieces of Google software as well. (The same behavior is in Python as well, but that's a bit unfair, since Python's IDNA encoding is effectively frozen to IDNA 2003.)

straße 6637
strasse 6637
xn--strae-oqa 33560

Discussion

Which way to go here is ultimately a policy change. The arguments for keeping transitional processing are:

  • Maintains strict compatibility with previous versions of Go
  • Maintains compatibility with Chrome, and Google products in general

The arguments for moving to nontransitional processing are:

  • Aligns with the rest of the world
  • Aligns with web standards

It appears to me that implementing things from web standards and first principles is part of the Go ethos, which would lean in favor of using nontransitional processing. However, maintaining compatibility is also a powerful motivation.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions