Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: can't handle Onion domains #42283

Closed
danger89 opened this issue Oct 29, 2020 · 6 comments
Closed

net/http: can't handle Onion domains #42283

danger89 opened this issue Oct 29, 2020 · 6 comments

Comments

@danger89
Copy link

@danger89 danger89 commented Oct 29, 2020

What version of Go are you using (go version)?

go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/melroy/.cache/go-build"
GOENV="/home/melroy/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/melroy/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/melroy/Documents/drone/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build611413724=/tmp/go-build -gno-record-gcc-switches"

What did you do?

While using the Go net/http package, try to connect to an valid onion address.

In my case I'm using Drone a CI tool in Docker written in Golang. Drone tries via Oauth2 to connect to the Git server (eg. Gitea), but the POST request on http://mydomain.onion/login/oauth/access_token for Gitea fails: no suitable address found. See code lines below for more details what Drone is doing.

Usage example: Code line of HTTP package usage in Drone.
Usage example 2: Call to Oauth2 package
--> Code line of HTTP usage in Oauth2 package of Drone

What did you expect to see?

I would expect that Golang is able to connect to onion domains by using the http package, WITHOUT errors.

What did you see instead?

dial tcp: address xxxx.onion: no suitable address found 

Output of Drone:

ERRO[0026] oauth: cannot exchange code: secret: Post "http://xxx.onion/login/oauth/access_token": dial tcp: address xxxx.onion: no suitable address found 
DEBU[0026] cannot authenticate user: Post "http://yyy.onion/login/oauth/access_token": dial tcp: address yyy.onion: no suitable address found 
DEBU[0026]                                               fields.time="2020-10-29T21:49:40Z" latency=1.611572ms method=GET remote="172.19.0.1:40842" request="/login?code=secret%3D&state=secret" request-id=secret

More reports

I'm not the only one suffering from this problem. Maybe not fully related, but somebody that tries to use go get by using a onion domain to fetch packages: https://www.reddit.com/r/techsupport/comments/981frd/need_help_with_golang_and_tor/
It could be touching the same code base in Go?

Thanks.

Regards,
Melroy van den Berg

@dmitshur dmitshur changed the title Golang HTTP package can't handle Onion domains net/http: can't handle Onion domains Oct 30, 2020
@dmitshur dmitshur added this to the Backlog milestone Oct 30, 2020
@dmitshur
Copy link
Member

@dmitshur dmitshur commented Oct 30, 2020

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Oct 31, 2020

Thank you for filing this issue/question @danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

Technically, given that .onion addresses aren't actual DNS names, they require you to connect to Tor first and then the Tor protocol will handle the tunnelling and direction to the appropriate resource -- thus DNS requests are ALWAYS going to fail if you don't have an appropriate reverse proxy. If you have your Tor compatible proxy running, you can create a reverse proxy in Go code by https://golang.org/pkg/net/http/httputil/#NewSingleHostReverseProxy or just set in your environment HTTP_PROXY as per https://golang.org/pkg/net/http/#RoundTripper.

From Go's stand point, and from pretty much every language and browser, without Tor-enabled support, resolving such domains won't work. Here is a reference to Facebook describing the problem and them announcing their .onion site https://www.facebook.com/notes/protect-the-graph/making-connections-to-facebook-more-secure/1526085754298237/

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

I am going to close this issue as non-actionable in 1 day, but in case someone else wants to chime in, or please feel free to correct me.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Nov 1, 2020

I agree with Odeke. We don't plan to support Tor natively in the standard library.

@odeke-em odeke-em closed this Nov 2, 2020
@danger89
Copy link
Author

@danger89 danger89 commented Nov 3, 2020

Thank you for filing this issue/question @danger89 and welcome to the Go project!
Thank you for the tag @dmitshur.

For a reference to a deliberate decision not to resolve them unless there is a SOCK5 proxy, please see #13705.

@odeke-em You know that Tor runs a SOCK5 proxy by default, on port 9050, so that is available..? In fact Tor by default will NOT provide you any HTTP proxy only sock5 (eg socks5h://127.0.0.1:9050).

Does GO support socks5h schema just like git and curl does?

@danger89
Copy link
Author

@danger89 danger89 commented Nov 3, 2020

Apparently it doesn't ;( proxyconnect tcp: dial tcp: lookup socks5h on <ip>: no such host

@danger89
Copy link
Author

@danger89 danger89 commented Nov 3, 2020

Using HTTPTunnelPort 8118 in Torrc and HTTP_PROXY=https://127.0.0.1:8118 will result in: proxyconnect tcp: dial tcp <ip:port>: connect: connection refused. since I Tor only provides CONNECT proxy not a GET proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.