Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: ProxyFromEnvironment no_proxy=localhost doesn't match empty Host #48325

Open
skiptomyliu opened this issue Sep 10, 2021 · 2 comments
Open
Labels
NeedsInvestigation
Milestone

Comments

@skiptomyliu
Copy link

@skiptomyliu skiptomyliu commented Sep 10, 2021

What version of Go are you using (go version)?

go version go1.16.6 darwin/amd64
go version go1.16.6 linux/amd64
go version go1.17.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes, also tested go 1.17.1

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/Users/dliu/src/go/bin/"
GOCACHE="/Users/dliu/Library/Caches/go-build"
GOENV="/Users/dliu/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/dliu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB="github.com/lyft/*,github.lyft.net/*"
GOOS="darwin"
GOPATH="/Users/dliu/go"
GOPRIVATE=""
GOROOT="/usr/local/Cellar/go/1.16.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16.6/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dliu/Desktop/gotest2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/hz/phr9x2hd2z16zyn6gcs8l86m0000gp/T/go-build4278321027=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ cat proxytest.go
package main

import (
        "log"
        "google.golang.org/grpc"
)

func main() {
        _, err := grpc.Dial(":9901", grpc.WithInsecure(), grpc.WithBlock())
        if err != nil {
                log.Fatal(err)
        }
}


$ GRPC_GO_LOG_SEVERITY_LEVEL=info GRPC_GO_LOG_VERBOSITY_LEVEL=99   no_proxy=localhost https_proxy=localhost:10001 /usr/local/go/bin/go run proxytest.go
INFO: 2021/09/10 10:36:42 [core] parsed scheme: ""
INFO: 2021/09/10 10:36:42 [core] scheme "" not registered, fallback to default scheme
INFO: 2021/09/10 10:36:42 [core] ccResolverWrapper: sending update to cc: {[{:9901  <nil> 0 <nil>}] <nil> <nil>}
INFO: 2021/09/10 10:36:42 [core] ClientConn switching balancer to "pick_first"
INFO: 2021/09/10 10:36:42 [core] Channel switches to new LB policy "pick_first"
INFO: 2021/09/10 10:36:42 [core] Subchannel Connectivity change to CONNECTING
INFO: 2021/09/10 10:36:42 [core] Subchannel picks a new address ":9901" to connect
INFO: 2021/09/10 10:36:42 [core] pickfirstBalancer: UpdateSubConnState: 0xc000191ac0, {CONNECTING <nil>}
INFO: 2021/09/10 10:36:42 [core] Channel Connectivity change to CONNECTING
WARNING: 2021/09/10 10:36:42 [core] grpc: addrConn.createTransport failed to connect to {:9901 localhost:9901 <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:10001: connect: connection refused". Reconnecting...

What did you expect to see?

Was expecting that because no_proxy=localhost is set, that the connection would obey and thus not CONNECT tunnel to the proxy. When a blank hostname is set, Go implicitly uses localhost. However, it seems I'm not able to no_proxy localhost if the hostname is set to blank. Explicitly setting to localhost:9901 resolves the issue.

What did you see instead?

The connection is attempting to use the proxy even though no_proxy=localhost.

Originally filed with grpc-go grpc/grpc-go#4719 (comment) but was directed here.

@thanm thanm added the NeedsInvestigation label Sep 10, 2021
@thanm thanm added this to the Backlog milestone Sep 10, 2021
@thanm
Copy link
Contributor

@thanm thanm commented Sep 10, 2021

@neild

@1000Delta
Copy link

@1000Delta 1000Delta commented Sep 12, 2021

Hello, I read some source code about your problem, I think this problem is caused by how grpc call the golang http.ProxyFromEnvironment.

For golang httpproxy.proxyForURL, you can see that if the argument reqURL.Scheme is "https", you will get the cfg.httpsProxy.

And in grpc, if your target is :9901, grpc will call the internal function transport.mapAddress, this function will create a struct http.Request and set inner url.URL.Scheme to "https", then will call the transport.httpProxyFromEnvironment, which is a variable of golang exported function http.ProxyFromEnvironment.

More about golang httpproxy.proxyForURL, it will call the cfg.useProxy, this method will check your host and ip, this also answer the question of why explicitly setting the grpc.Dial target to localhost:9901 can resolve this issue, it matches localhost before the :. However, only setting the port like :9901 will not let cfg.useProxy return false. So you will get your httpsProxy.

You can debug and set a breakpoint to mapAddress function, you will see the call will get your https_proxy environment.

I hope this is helpful.

@seankhliao seankhliao changed the title no_proxy is not respected when empty host is used net/http: ProxyFromEnvironment no_proxy=localhost doesn't match empty Host Feb 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

3 participants