net/http/httputil: Grpc over Go Proxy brings always connection error (help wanted)

[idefixcert]

What version of Go are you using (go version)?

go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

darvin

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/chris/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/chris/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pp/hltp86tn7nn81wfpqh0nh8k00000gn/T/go-build247331372=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a simple setup: no proxy server configured
grpcClient -calls-> go proxy -forward request to-> grpcServer

The grpcServer runs without tls.
The go proxy (code below) allows tls and no tls
I call grpcClient with and without tls.

package main

import (
	"flag"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"os"
)

// argument parser
var (
	upstream    string
	httpsSocket string
	httpsCert   string
	httpsKey    string
)

func init() {
	flag.StringVar(&upstream, "upstream", "", "upstream (http://<ip>:<port>)")
	flag.StringVar(&httpsSocket, "socket", ":9090", "local socket")
	flag.StringVar(&httpsCert, "cert", "server.crt", "ssl cert")
	flag.StringVar(&httpsKey, "key", "server.key", "key")
	flag.Parse()
}

// main function
func main() {
	upstream = "http://localhost:10000"
	log.SetPrefix("[proxy] ")
	log.SetOutput(os.Stdout)
	if upstream == "" {
		log.Fatal("ERROR: missing argument upstream ")
	}
	url, _ := url.Parse(upstream)
	proxy := &Upstream{target: url, proxy: httputil.NewSingleHostReverseProxy(url)}

	mux := http.NewServeMux()
	mux.HandleFunc("/", proxy.handle)
        //Only one line shout be commented tls/notls
	log.Fatal(http.ListenAndServeTLS(httpsSocket, httpsCert, httpsKey, mux))
	//log.Fatal(http.ListenAndServe(httpsSocket, mux))

}

// Upstream ...
type Upstream struct {
	target *url.URL
	proxy  *httputil.ReverseProxy
}

func (p *Upstream) handle(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("X-Forwarded-For", r.Host)
	p.proxy.ServeHTTP(w, r)
}

What did you expect to see?

1.) If I use the proxy for http (Browser) it works.
2.) If I run without proxy, the call to the server returns.
3.) If I run with proxy, the call to the server returns.

What did you see instead?

1.) works
2.) works
3.) Without tls the client brings the following output:

2019/01/25 00:33:23 Getting feature for point (409146138, -746188906)
2019/01/25 00:33:23 &{0xc000138000}.GetFeatures(_) = _, rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>: 
exit status 1

No output on proxy and server.

With tls I get:
Client:

2019/01/25 00:41:55 Getting feature for point (409146138, -746188906)
2019/01/25 00:41:55 &{0xc000162600}.GetFeatures(_) = _, rpc error: code = Unavailable desc = Bad Gateway: 
exit status 1

Proxy:

[proxy] 2019/01/25 00:41:55 http: proxy error: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x00\x04\x00\x00\x00\x00\x00"

[carl-mastrangelo]

Not commenting on this issue (which may be a real bug), but httputil's reverse proxy methods need some help to work with gRPC. I have created a wrapper library to work with it (and use it myself):

https://github.com/carl-mastrangelo/h2c/

There are examples on how to use this with reverse proxy.

[bcmills]

CC @bradfitz for net/http/httputil, but I would suggest that you follow up in a forum with more GRPC-specific knowledge.

(http://golang.org/wiki/Questions has some starting points, or you could try the GRPC venues linked from https://grpc.io/community/.)

[idefixcert]

ok, with the help from @carl-mastrangelo I was able to fix the error in my proxy:

Here the code maybe someone helps.

Thanx for your help.

// argument parser
var (
	upstream    string
	httpsSocket string
	httpsCert   string
	httpsKey    string
)

func init() {
	flag.StringVar(&upstream, "upstream", "", "upstream (http://<ip>:<port>)")
	flag.StringVar(&httpsSocket, "socket", ":9090", "local socket")
	flag.StringVar(&httpsCert, "cert", "server.crt", "ssl cert")
	flag.StringVar(&httpsKey, "key", "server.key", "key")
	flag.Parse()
}

// main function
func main() {
	upstream = "http://localhost:10000"
	log.SetPrefix("[proxy] ")
	log.SetOutput(os.Stdout)
	if upstream == "" {
		log.Fatal("ERROR: missing argument upstream ")
	}
	url, _ := url.Parse(upstream)
	proxy := &Upstream{target: url, proxy: httputil.NewSingleHostReverseProxy(url)}

	mux := http.NewServeMux()
	mux.HandleFunc("/", proxy.handle)
	log.Fatal(http.ListenAndServeTLS(httpsSocket, httpsCert, httpsKey, mux))

}

// Upstream ...
type Upstream struct {
	target *url.URL
	proxy  *httputil.ReverseProxy
}

func (p *Upstream) handle(w http.ResponseWriter, r *http.Request) {
	log.Println("here")
	w.Header().Set("X-Forwarded-For", r.Host)
	p.proxy.Transport =
		&http2.Transport{
			AllowHTTP: true,
			DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
				ta, err := net.ResolveTCPAddr(network, addr)
				if err != nil {
					return nil, err
				}
				return net.DialTCP(network, nil, ta)
			},
		}
	p.proxy.ServeHTTP(w, r)
}