net/http: Server with TLS consumes "Upgrade" header

[ssttevee]

What version of Go are you using (go version)?

$ go version
go version go1.12rc1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/steve/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/steve/.gvm/pkgsets/go1.12rc1/global"
GOPROXY=""
GORACE=""
GOROOT="/home/steve/.gvm/gos/go1.12rc1"
GOTMPDIR=""
GOTOOLDIR="/home/steve/.gvm/gos/go1.12rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/steve/http-test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build314515568=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go run the following code followed by a curl with a Upgrade header.

package main

import (
	"log"
	"net/http"
)

func main() {
	var handler http.HandlerFunc = func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte(r.Header.Get("Upgrade")))
	}

	log.Fatal(http.ListenAndServeTLS(":8443", "rsa.crt", "rsa.key", handler))
}

What did you expect to see?

$ curl -k -i --header "Upgrade: websocket" "https://localhost:8443/"
HTTP/2 200
content-length: 9

websocket

What did you see instead?

$ curl -k -i --header "Upgrade: websocket" "https://localhost:8443/"
HTTP/2 200
content-length: 0

[bcmills]

CC @FiloSottile @bradfitz

[odeke-em]

Hello @ssttevee, thank you for filing this issue and welcome to the Go project!

So firstly, cURL will fix the headers for you and include "Connection": "upgrade" after seeing "Upgrade": "Websocket" which it fixed from "Upgrade": "websocket".
so your request on the wire then becomes

GET / HTTP/2
Host: localhost:8844
User-Agent: curl/7.54.0
Accept: */*
Upgrade:Websocket
Connection:upgrade

Now since "Connection" is a hop-by-by and connection-specific header but also not supported by HTTP/2.0 (well for Go, except only if "Connection": "close" that means a graceful shutdown), including it in an HTTP/2.0 response violates the spec at Section 8.1.2.2
which says

HTTP/2 does not use the Connection header field to indicate connection-specific header fields; in this protocol, connection-specific metadata is conveyed by other means. An endpoint MUST NOT generate an HTTP/2 message containing connection-specific header fields; any message containing connection-specific header fields MUST be treated as malformed (Section 8.1.2.6).

The only exception to this is the TE header field, which MAY be present in an HTTP/2 request; when it is, it MUST NOT contain any value other than "trailers".

This means that an intermediary transforming an HTTP/1.x message to HTTP/2 will need to remove any header fields nominated by the Connection header field, along with the Connection header field itself. Such intermediaries SHOULD also remove other connection-specific header fields, such as Keep-Alive, Proxy-Connection, Transfer-Encoding, and Upgrade, even if they are not nominated by the Connection header field.

as you can see from above, the spec mandates that a valid HTTP/2.0 server remove connection-specific headers and in this case "Connection" and "Upgrade" are pruned out.

If you use Go's HTTP client, it will automatically instead use an HTTP/1.1 connection and your HTTP/1.1 response will include the headers back such as

HTTP/1.1 200 OK
Content-Length: 137
Content-Type: text/plain; charset=utf-8
Date: Wed, 13 Feb 2019 14:07:11 GMT

GET / HTTP/1.1
Host: localhost:8844
Accept-Encoding: gzip
Connection: upgrade
Upgrade: Websocket
User-Agent: Go-http-client/1.1

Thus I believe this is working as intended and not a bug.

Kindly paging @bradfitz to help me double check my assessment.

[bradfitz]

Yeah, what @odeke-em said. You can see the same behavior on http2.golang.org which is running Go 1.11, not Go 1.12rc1:

dev:~ $ curl -k -i --http1.1 --header "Upgrade: websocket" https://http2.golang.org/reqinfo
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Wed, 13 Feb 2019 14:57:39 GMT
Content-Length: 902

Method: GET
Protocol: HTTP/1.1
Host: http2.golang.org
RemoteAddr: xxx
RequestURI: "/reqinfo"
URL: &url.URL{Scheme:"", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/reqinfo", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}
Body.ContentLength: 0 (-1 means unknown)
Close: false (relevant for HTTP/1 only)
TLS: &tls.ConnectionState{Version:0x303, HandshakeComplete:true, DidResume:false, CipherSuite:0xc02b, NegotiatedProtocol:"", NegotiatedProtocolIsMutual:true, ServerName:"http2.golang.org", PeerCertificates:[]*x509.Certificate(nil), VerifiedChains:[][]*x509.Certificate(nil), SignedCertificateTimestamps:[][]uint8(nil), OCSPResponse:[]uint8(nil), ekm:(func(string, []uint8, int) ([]uint8, error))(0x60a540), TLSUnique:[]uint8{0xd3, 0xc8, 0xa, 0xe5, 0x37, 0x84, 0x8d, 0x9e, 0x76, 0xdd, 0x3a, 0x37}}

Headers:
Accept: */*
Upgrade: websocket
User-Agent: curl/7.52.1

vs

dev:~ $ curl -k -i --http2 --header "Upgrade: websocket" https://http2.golang.org/reqinfo
HTTP/2 200 
content-type: text/plain
content-length: 885
date: Wed, 13 Feb 2019 14:57:41 GMT

Method: GET
Protocol: HTTP/2.0
Host: http2.golang.org
RemoteAddr: xxx
RequestURI: "/reqinfo"
URL: &url.URL{Scheme:"", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/reqinfo", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}
Body.ContentLength: 0 (-1 means unknown)
Close: false (relevant for HTTP/1 only)
TLS: &tls.ConnectionState{Version:0x303, HandshakeComplete:true, DidResume:false, CipherSuite:0xc02b, NegotiatedProtocol:"h2", NegotiatedProtocolIsMutual:true, ServerName:"http2.golang.org", PeerCertificates:[]*x509.Certificate(nil), VerifiedChains:[][]*x509.Certificate(nil), SignedCertificateTimestamps:[][]uint8(nil), OCSPResponse:[]uint8(nil), ekm:(func(string, []uint8, int) ([]uint8, error))(0x60a540), TLSUnique:[]uint8{0x64, 0x96, 0xea, 0x85, 0x6d, 0x24, 0xab, 0x79, 0x45, 0x18, 0x78, 0x5f}}

Headers:
Accept: */*
User-Agent: curl/7.52.1

I'm closing this as not a bug, but let us know if there's some other problem you're hitting that led you to debugging and suspecting this.