Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/http2: Transport Proxy field isn't always respected on TCP level #25620

Closed
plambein opened this issue May 29, 2018 · 5 comments
Closed

x/net/http2: Transport Proxy field isn't always respected on TCP level #25620

plambein opened this issue May 29, 2018 · 5 comments

Comments

@plambein
Copy link

@plambein plambein commented May 29, 2018

What version of Go are you using (go version)?

go1.10.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="..."
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="..."
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.2/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_j/6wqhr8r57g34l96w0vt17bww0000gp/T/go-build671837991=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I build an HTTP/2.0 server and 2 TCP proxy servers. Then I executed 20 concurrent requests (with an HTTP/2.0 transport) to the server, alternating the proxy servers to go through. The transport has been configured to disable keepalives.
The server prints out the http.Request.RemoteAddress of the request and it's ID. (The ID determines which proxy server should be selected: odd or even number). So we can see if a connection has been reused or not.

The code can be found here: https://play.golang.org/p/MNtzLK0KX_T
However it won't run in the playground, since the "golang.org/x/net/http2" package is used.

What did you expect to see?

When commenting out the http2.ConfigureTransport, the output becomes:

Address: 127.0.0.1:49934 - ID: 2
Address: 127.0.0.1:49936 - ID: 1
Address: 127.0.0.1:49932 - ID: 5
Address: 127.0.0.1:49937 - ID: 13
Address: 127.0.0.1:49933 - ID: 4
Address: 127.0.0.1:49938 - ID: 3
Address: 127.0.0.1:49951 - ID: 8
Address: 127.0.0.1:49935 - ID: 0
Address: 127.0.0.1:49949 - ID: 7
Address: 127.0.0.1:49941 - ID: 9
Address: 127.0.0.1:49940 - ID: 12
Address: 127.0.0.1:49946 - ID: 18
Address: 127.0.0.1:49939 - ID: 19
Address: 127.0.0.1:49943 - ID: 10
Address: 127.0.0.1:49942 - ID: 14
Address: 127.0.0.1:49944 - ID: 16
Address: 127.0.0.1:49948 - ID: 15
Address: 127.0.0.1:49945 - ID: 17
Address: 127.0.0.1:49947 - ID: 11
Address: 127.0.0.1:49950 - ID: 6

Which means every request was done through it's own connection. Nothing has been reused. This was what I expected to see for HTTP/2.0 as well.

What did you see instead?

This is the output of the program:

Address: 127.0.0.1:64247 - ID: 0
Address: 127.0.0.1:64248 - ID: 5
Address: 127.0.0.1:64249 - ID: 12
Address: 127.0.0.1:64250 - ID: 13
Address: 127.0.0.1:64251 - ID: 8
Address: 127.0.0.1:64252 - ID: 1
Address: 127.0.0.1:64253 - ID: 6
Address: 127.0.0.1:64254 - ID: 19
Address: 127.0.0.1:64254 - ID: 10
Address: 127.0.0.1:64256 - ID: 14
Address: 127.0.0.1:64257 - ID: 7
Address: 127.0.0.1:64258 - ID: 2
Address: 127.0.0.1:64258 - ID: 15
Address: 127.0.0.1:64260 - ID: 11
Address: 127.0.0.1:64261 - ID: 16
Address: 127.0.0.1:64262 - ID: 3
Address: 127.0.0.1:64263 - ID: 18
Address: 127.0.0.1:64264 - ID: 9
Address: 127.0.0.1:64265 - ID: 4
Address: 127.0.0.1:64266 - ID: 17

Some of these requests have been handled by the same connection. E.g. requests with ID 10 and 19, and requests with ID 2 and 15 have been send through the same remote address, which means they used the same proxy server as well, even though the odd ID's should use a different proxy server from the even ID's.

So even though a proxy server was assigned on HTTP level, through the http.Transport.Proxy, it isn't always respected on TCP level, since a previous connection gets reused.

EDIT @vagruchi pointed out an issue with the previous version of the issue, where the proxy server wasn't actually used. This has been fixed in the current implementation and the output has been updated.

@agnivade agnivade added this to the Go1.11 milestone May 30, 2018
@vagruchi

This comment has been minimized.

Copy link

@vagruchi vagruchi commented May 30, 2018

Try to unmute errors:

	proxy1URL, err := url.Parse("127.0.0.1:8001")
	fmt.Println(err)

Output is:
parse 127.0.0.1:8001: first path segment in URL cannot contain colon
proxy1URL and proxy2URL is nil.
If Proxy returns nil, no proxy is used.

@plambein

This comment has been minimized.

Copy link
Author

@plambein plambein commented May 31, 2018

You are right, my bad. There seems to be an issue with the proof of concept. The issue is real though. I'll look into fixing the example.

@plambein

This comment has been minimized.

Copy link
Author

@plambein plambein commented Jun 5, 2018

@vagruchi The issue has been fixed in the version above.

@meirf

This comment has been minimized.

Copy link
Contributor

@meirf meirf commented Jun 11, 2018

I think there are still problems with the proof of concept. Using env var GODEBUG=http2debug=1 shows a entries like:

read tcp 127.0.0.1:54530->127.0.0.1:8001: use of closed network connection
read tcp 127.0.0.1:54532->127.0.0.1:8002: use of closed network connection

For context, please see this discussion.

The behavior you say is unexpected of connection reuse is actually happening when there is not an error yet - before the connections are getting closed due to the above error. So I expect that when the error is fixed, you'll see a lot more reuse. I encourage you to fix the error to confirm, but I think the underlying problem is more clearly highlighted by #25793.

@bradfitz

This comment has been minimized.

Copy link
Contributor

@bradfitz bradfitz commented Jul 13, 2018

Sounds like this bug has timed out and might be a bug in the repro. Or it might be a duplicate of #25793. In any case, closing in favor of #25793.

@bradfitz bradfitz closed this Jul 13, 2018
@golang golang locked and limited conversation to collaborators Jul 13, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.