Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: missing status in a proxied HTTPS responses after CONNECT request causes index out of range runtime panic #21701

Closed
soluchok opened this issue Aug 30, 2017 · 11 comments

Comments

Projects
None yet
4 participants
@soluchok
Copy link
Contributor

commented Aug 30, 2017

What version of Go are you using (go version)?

go version go1.9 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/gopath/"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build419562759=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

My code is:

        proxyURL, _ := url.Parse(fmt.Sprintf("http://%s", proxy))
	client := &http.Client{
		Timeout: 5 * time.Second,
		Transport: &http.Transport{
			Proxy: http.ProxyURL(proxyURL),
			DialContext: (&net.Dialer{
				Timeout: 5 * time.Second,
			}).DialContext,
			TLSHandshakeTimeout: 5 * time.Second,
			DisableKeepAlives:   true,
		},
	}
	r, _ := http.NewRequest("POST", BaseUrl, nil)
	resp, _ := client.Do(r)

What did you expect to see?

Anything, but no panic in runtime :)

What did you see instead?

I have got the error in production

panic: runtime error: index out of range

goroutine 2895221 [running]:
net/http.(*Transport).dialConn(0xc42010a4b0, 0xe55ec0, 0xc4200140a8, 0xc420217c80, 0xc4216901b0, 0x5, 0xc4206d8e60, 0xe, 0xc421862d20, 0x5ac3e4, ...)
	/usr/local/go/src/net/http/transport.go:1128 +0x2090
net/http.(*Transport).getConn.func4(0xc42010a4b0, 0xe55ec0, 0xc4200140a8, 0xc420766510, 0xc421b96ae0)
	/usr/local/go/src/net/http/transport.go:943 +0x78
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:942 +0x393

I fixed it, but I cannot to imitate a situation for test.
The problem is when Status is '400' and StatusCode is '400' (that's a real, I checked in production)
See https://go-review.googlesource.com/c/go/+/59990/

@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

@soluchok seems like it might be a malformed response from the external server referenced in your code by proxy. Would you mind say using curl to output the response information and headers, say?

curl -i $URL

where $URL is the proxy URL or the actual URL?

@soluchok

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2017

@odeke-em It happens very rarely, but it does happen.
I think it's something like that:

HTTP/1.1 400
Date: Wed, 30 Aug 2017 19:09:27 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Last-Modified: Wed, 30 Aug 2017 19:02:02 GMT
Vary: Accept-Encoding
Server: cloudflare-nginx

When I caught it the status was '400' but should be '400 Bad request'

@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

Yeah I was thinking that too but that path gets caught and reported with a "malformed HTTP response" when missing that status itself. We also have tests in net/http to test for this kind of thing.

Here is code to try to repro it, but in vain, trying as much to replicate the server as well as your client code at gist https://gist.github.com/odeke-em/876d24325d16b3885388f7f0dae9611d or inlined

Server

package main

import (
	"flag"
	"fmt"
	"log"
	"net"
)

func main() {
	var port int
	flag.IntVar(&port, "port", 8877, "the port to listen on")
	flag.Parse()

	addr := fmt.Sprintf(":%d", port)
	log.Printf("server listening at %q\n", addr)
	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatal("listen: %v", err)
	}

	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Printf("listen err: %v", err)
			continue
		}
		go handleConn(conn)
	}
}

var malformed = []byte(
`HTTP/1.1 400
Date: Wed, 30 Aug 2017 19:09:27 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 10
Connection: close
Last-Modified: Wed, 30 Aug 2017 19:02:02 GMT
Vary: Accept-Encoding
Server: cloudflare-nginx` + "\r\n\r\nAloha Olaa")

func handleConn(conn net.Conn) {
	conn.Write(malformed)
	conn.Close()
}

Client

package main

import (
	"flag"
	"fmt"
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/url"
	"time"
)

func main() {
	var theURL string
	flag.StringVar(&theURL, "url", "http://localhost:8877", "the server address")
	flag.Parse()

	proxyURL, _ := url.Parse(theURL)
	client := &http.Client{
		Timeout: 5 * time.Second,
		Transport: &http.Transport{
			Proxy: http.ProxyURL(proxyURL),
			DialContext: (&net.Dialer{
				Timeout: 5 * time.Second,
			}).DialContext,
			TLSHandshakeTimeout: 5 * time.Second,
			DisableKeepAlives:   true,
		},
	}
	r, _ := http.NewRequest("POST", "https://golang.org/", nil)
	res, err := client.Do(r)
	if err != nil {
		log.Fatalf("res: %v", err)
	}
	wire, err := httputil.DumpResponse(res, true)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("wire: %s\n", wire)
}

and it mimicks that response that you got back, I also setup the proxy in the transport for the client.

I'll also /cc @tombergan in case he might have insights.

@odeke-em odeke-em changed the title net/http: panic: runtime error: index out of range net/http: missing status in a response at times causes index out of range runtime panic Aug 30, 2017

@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

NVM, am able to reproduce it now with the server

@tombergan

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2017

This is definitely a bug:

f := strings.SplitN(resp.Status, " ", 2)

We cannot guarantee there's actually a space there, so we shouldn't access f[1] without checking len(f).

@odeke-em could you reproduce it with the code from this comment? Looks like it only happens when trying to proxy an HTTPS URL, which is transformed into a CONNECT request.

@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

@tombergan yap, with the code I posted I am able to reproduce it

$ go run client.go 
panic: runtime error: index out of range

goroutine 7 [running]:
net/http.(*Transport).dialConn(0xc4200f8000, 0x129fd80, 0xc42001e048, 0xc4200f6000, 0x1271242, 0x5, 0xc42001e460, 0xe, 0x0, 0x0, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/net/http/transport.go:1128 +0x1fa4
net/http.(*Transport).getConn.func4(0xc4200f8000, 0x129fd80, 0xc42001e048, 0xc42007ae10, 0xc420020600)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/net/http/transport.go:943 +0x78
created by net/http.(*Transport).getConn
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/net/http/transport.go:942 +0x355
exit status 2
@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

Am now working on a minimal test case that we can then put in @soluchok's CL, review and then sail the ship to merge.

@soluchok

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2017

@odeke-em Thanks)

@soluchok soluchok closed this Aug 30, 2017

@tombergan

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2017

Not fixed yet.

@tombergan tombergan reopened this Aug 30, 2017

@gopherbot

This comment has been minimized.

Copy link

commented Aug 30, 2017

Change https://golang.org/cl/59990 mentions this issue: net/http: Fixes #21701 fix panic when status empty

@odeke-em

This comment has been minimized.

@odeke-em odeke-em changed the title net/http: missing status in a response at times causes index out of range runtime panic net/http: missing status in a proxied HTTPS responses after CONNECT request causes index out of range runtime panic Aug 30, 2017

@odeke-em odeke-em added this to the Go1.10 milestone Aug 30, 2017

@gopherbot gopherbot closed this in eb69581 Oct 16, 2017

@golang golang locked and limited conversation to collaborators Oct 16, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.