Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: cannot set custom/special pseudo-headers when using HTTP/2 #32763

Open
jaddr2line opened this issue Jun 25, 2019 · 11 comments

Comments

@jaddr2line
Copy link

@jaddr2line jaddr2line commented Jun 25, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jadenw/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jadenw/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build648789117=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I attempted to implement websockets over HTTP/2, AKA RFC 8441, with net/http.

What did you expect to see?

Some mechanism of setting :protocol as defined in section 5 of RFC 8441.

What did you see instead?

There is no way to specify extra pseudo-headers in a request with net/http, at least that I can find in the header encoding code.

I am really not sure what the best way to solve this would be, or whether it is fully worthwhile. However, I would really like to be able to use a user-provided *http.Client for this, and this is the only thing I have not yet found a workaround for.

Clarification: pseudo-headers do not satisfy httpguts.ValidHeaderFieldName and therefore cannot be passed in with the regular headers

@jaddr2line

This comment has been minimized.

Copy link
Author

@jaddr2line jaddr2line commented Jun 25, 2019

Also, a link to what I have currently of the client side of the HTTP/2 handshake: https://github.com/jadr2ddude/exp/blob/1ed5641ad76ca5761af3a652406bf49b7cac2ce3/ws/handshake.go#L248

@andybons andybons changed the title Setting Custom/Special Pseudo-Headers for HTTP/2 net/http: cannot set custom/special pseudo-headers when using HTTP/2 Jun 25, 2019
@andybons andybons added this to the Unplanned milestone Jun 25, 2019
@andybons

This comment has been minimized.

Copy link
Member

@andybons andybons commented Jun 25, 2019

@jaddr2line jaddr2line mentioned this issue Jun 26, 2019
@mikedanese

This comment has been minimized.

Copy link
Contributor

@mikedanese mikedanese commented Jul 5, 2019

This is somewhat related to #27244

@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Jul 25, 2019

@bradfitz how about we automatically convert HTTP/1.1 Upgrade requests into HTTP/2 if the peer supports it in *http.Transport? We'd have to perform an initial HTTP/2 request on the endpoint to check if it supports RFC 8441, cache that and then from there we can automatically decide whether or not to do HTTP/1.1 or HTTP/2 for subsequent upgrades.

Would mean my WebSocket library basically gets client side HTTP/2 WebSockets for free and httputil.ReverseProxy can proxy RFC 8441 connect requests.

@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Jul 25, 2019

For the server side

  1. We can just reuse http.Hijacker and return a net.Conn that tries its best to wrap around a HTTP/2 stream, unfortunately we'd have to allocate a buffo.ReadWriter every time but that shouldn't be too big a deal, most apps will want a bufio.ReadWriter when reading/writing to the stream anyway. We could add the Connection and Upgrade headers to RFC 8441 protocol connect requests too so that net/http hijacking servers all get this for free.
  2. Then once the above is implemented, we just need to always set the SETTINGS_ENABLE_CONNECT_PROTOCOL HTTP/2 setting parameter to indicate we support RFC 8441
@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Jul 25, 2019

There was previous discussion of this at #26937 (comment).

@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Oct 12, 2019

@jadr2ddude, thank you for filing this issue! As @nhooyr mentioned, so if you are implementing Websockets I believe the first step that you want to do is to hijack the connection and then if you know you are using HTTP/2, you can use the golang.org/x/net/http2 package to create the framer and send over the respective headers by yourself. IMHO we shouldn't be changing net/http to make that change here. Please feel free to reopen though if I've missed something, thank you!

@odeke-em odeke-em closed this Oct 12, 2019
@mikedanese

This comment has been minimized.

Copy link
Contributor

@mikedanese mikedanese commented Oct 12, 2019

@odeke-em how would you use the framer without having to rewrite or fork most of net/http? This needs both client and server support.

@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Oct 12, 2019

You can’t hijack the connection with http/2. So this needs to be reopened.

@bradfitz bradfitz reopened this Oct 12, 2019
@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Oct 12, 2019

My apologies @jadr2ddude @mikedanese @nhooyr for the brainless close and non-responsiveness for the correction, I posted my response right before a long dinner. Thanks for reopening it, Brad!

@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Oct 13, 2019

@bradfitz I'd like to prepare some CLs for this in the coming weeks, could you please review my proposal above?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.