net/http: can't handle Onion domains

[danger89]

What version of Go are you using (go version)?

go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/melroy/.cache/go-build"
GOENV="/home/melroy/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/melroy/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/melroy/Documents/drone/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build611413724=/tmp/go-build -gno-record-gcc-switches"

What did you do?

While using the Go net/http package, try to connect to an valid onion address.

In my case I'm using Drone a CI tool in Docker written in Golang. Drone tries via Oauth2 to connect to the Git server (eg. Gitea), but the POST request on http://mydomain.onion/login/oauth/access_token for Gitea fails: no suitable address found. See code lines below for more details what Drone is doing.

Usage example: Code line of HTTP package usage in Drone.
Usage example 2: Call to Oauth2 package
--> Code line of HTTP usage in Oauth2 package of Drone

What did you expect to see?

I would expect that Golang is able to connect to onion domains by using the http package, WITHOUT errors.

What did you see instead?

dial tcp: address xxxx.onion: no suitable address found 

Output of Drone:

ERRO[0026] oauth: cannot exchange code: secret: Post "http://xxx.onion/login/oauth/access_token": dial tcp: address xxxx.onion: no suitable address found 
DEBU[0026] cannot authenticate user: Post "http://yyy.onion/login/oauth/access_token": dial tcp: address yyy.onion: no suitable address found 
DEBU[0026]                                               fields.time="2020-10-29T21:49:40Z" latency=1.611572ms method=GET remote="172.19.0.1:40842" request="/login?code=secret%3D&state=secret" request-id=secret

More reports

I'm not the only one suffering from this problem. Maybe not fully related, but somebody that tries to use go get by using a onion domain to fetch packages: https://www.reddit.com/r/techsupport/comments/981frd/need_help_with_golang_and_tor/
It could be touching the same code base in Go?

Thanks.

Regards,
Melroy van den Berg

[dmitshur]

CC @odeke-em, @bradfitz, @empijei.

[odeke-em]

Thank you for filing this issue/question @danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

Technically, given that .onion addresses aren't actual DNS names, they require you to connect to Tor first and then the Tor protocol will handle the tunnelling and direction to the appropriate resource -- thus DNS requests are ALWAYS going to fail if you don't have an appropriate reverse proxy. If you have your Tor compatible proxy running, you can create a reverse proxy in Go code by https://golang.org/pkg/net/http/httputil/#NewSingleHostReverseProxy or just set in your environment HTTP_PROXY as per https://golang.org/pkg/net/http/#RoundTripper.

From Go's stand point, and from pretty much every language and browser, without Tor-enabled support, resolving such domains won't work. Here is a reference to Facebook describing the problem and them announcing their .onion site https://www.facebook.com/notes/protect-the-graph/making-connections-to-facebook-more-secure/1526085754298237/

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

I am going to close this issue as non-actionable in 1 day, but in case someone else wants to chime in, or please feel free to correct me.

[bradfitz]

I agree with Odeke. We don't plan to support Tor natively in the standard library.

[danger89]

Thank you for filing this issue/question @danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

@odeke-em You know that Tor runs a SOCK5 proxy by default, on port 9050, so that is available..? In fact Tor by default will NOT provide you any HTTP proxy only sock5 (eg socks5h://127.0.0.1:9050).

Does GO support socks5h schema just like git and curl does?

[danger89]

Apparently it doesn't ;( proxyconnect tcp: dial tcp: lookup socks5h on <ip>: no such host

[danger89]

Using HTTPTunnelPort 8118 in Torrc and HTTP_PROXY=https://127.0.0.1:8118 will result in: proxyconnect tcp: dial tcp <ip:port>: connect: connection refused. since I Tor only provides CONNECT proxy not a GET proxy.