Add http/2 cleartext server (h2c) support

[Zetanova]

The normal case is always to encrypt http2 network connections,
but since there are cases in gRPC and service-meshes where the upstream
is over an unix domain stocket (UDS), http/2 cleartext (h2c) would be required.

In a TLS connection the client can negate the h2 protocol with the help of TLS-ALPN header.

But in the case of http/2 cleartext (h2c) there are two different methods:

1. h2c upgrade
2. h2c prior knowledge

The golang native server can negate it already by inserting the h2c middleware Handler from golang.org/x/net/http2/h2c with h2c.NewHandler(handler, h2s)
h2c-golang-example

By simply extending the listener factory caddy2 could then serve
h2c connections by h2c update and also at the same time by h2c prior knowledge

I think, the h2c-handler can be added here:

caddy/modules/caddyhttp/app.go

Lines 188 to 217 in 84c729e

	// set up each listener modifier
	if srv.ListenerWrappersRaw != nil {
	vals, err := ctx.LoadModule(srv, "ListenerWrappersRaw")
	if err != nil {
	return fmt.Errorf("loading listener wrapper modules: %v", err)
	}
	var hasTLSPlaceholder bool
	for i, val := range vals.([]interface{}) {
	if _, ok := val.(*tlsPlaceholderWrapper); ok {
	if i == 0 {
	// putting the tls placeholder wrapper first is nonsensical because
	// that is the default, implicit setting: without it, all wrappers
	// will go after the TLS listener anyway
	return fmt.Errorf("it is unnecessary to specify the TLS listener wrapper in the first position because that is the default")
	}
	if hasTLSPlaceholder {
	return fmt.Errorf("TLS listener wrapper can only be specified once")
	}
	hasTLSPlaceholder = true
	}
	srv.listenerWrappers = append(srv.listenerWrappers, val.(caddy.ListenerWrapper))
	}
	// if any wrappers were configured but the TLS placeholder wrapper is
	// absent, prepend it so all defined wrappers come after the TLS
	// handshake; this simplifies logic when starting the server, since we
	// can simply assume the TLS placeholder will always be there
	if !hasTLSPlaceholder && len(srv.listenerWrappers) > 0 {
	srv.listenerWrappers = append([]caddy.ListenerWrapper{new(tlsPlaceholderWrapper)}, srv.listenerWrappers...)
	}
	}

If required http1 coud be disabled to accept only h2c-prior-knowledge,
but then caddy would need to bind the http2 server directly to the tcp-connection like

l, err := net.Listen("tcp", "0.0.0.0:1010")
checkErr(err, "while listening")

fmt.Printf("Listening [0.0.0.0:1010]...\n")
for {
    conn, err := l.Accept()
    checkErr(err, "during accept")

    server.ServeConn(conn, &http2.ServeConnOpts{
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintf(w, "Hello, %v, http: %v", r.URL.Path, r.TLS == nil)
        }),
    })
}

Because the h2c-handler covers both options, this feature could also be omitted.

Maybe the h2c handler can be added by extending the
JSON Config Structure › apps › http › servers › listener_wrappers

Demo Caddyfile for UDS

http://

bind unix//tmp/caddy.sock

reverse_proxy {
    to unix//temp/upstream.sock
    transport http {
        versions h2c
    }
}

Demo Caddyfile for TCP

http://

bind 127.0.0.1

reverse_proxy {
    to http://127.0.0.1:2012
    transport http {
        versions h2c
    }
}

[mholt]

To clarify:

service-meshes where the upstream is over an unix domain stocket (UDS), http/2 cleartext (h2c) would be required.

The socket used (UDS or otherwise) has nothing to do with whether TLS is used. It's a totally separate thing.

@Zetanova Also, I don't believe we need to use the method with server.ServeConn() nor mess with listeners at all, since the h2c.NewHandler() method supports both upgrade and prior knowledge.

[Zetanova]

@mholt yes, the method with server.ServeConn()should not be required.
Most UDS stream setups are unencrypted.

[mholt]

@Zetanova Alrighty, I have pushed experimental support for h2c server in the h2c branch:
b61e1a2

It is enabled by setting "insecure_h2c": true in the server struct (so, for example, adjacent to "listen"). (Sorry, didn't have time to whip up Caddyfile support yet.)

As before, I haven't tested it; will you try it out and see how it goes?

[Zetanova]

@mholt thx a lot!

looks good, the listener is working.

But the gRPC-response error/bug from the other new "h2c upstream" feature still breaks the gRPC communication. I hope that it will work with an upstream over normal tcp-h2

Test setup:
[gloang gRPC client] <- uds-h2c -> [caddy] <- tcp-h2c -> [kestrel gRPC server]

[mholt]

@Zetanova

But the gRPC-response error/bug from the other new "h2c upstream" feature still breaks the gRPC communication.

Do you mean this one?

[Zetanova]

@mholt No, this one

the gRPC Response in caddy gets somehow to transfered correctly back to the gRPC-client
Behavior the same with [golang gRPC client] <h2c> [caddy] <h2c> [dotnet gRPC server] and [dotnet gRPC client] <h2|h2c> [caddy] <h2c> [dotnet gRPC server]

i will test now [dotnet gRPC client] <h2|h2c> [caddy] <h2> [dotnet gRPC server]

[mholt]

Gotcha...

I'm afraid I don't know enough about gRPC to debug that one :( I'll need some hints to make much more progress. (Hopefully it is simple!)

[Zetanova]

@mholt gRPC info both gRPC clients (dotnet and golang) complaining about a missing response status code

Listening with h2c looks like to work now!
So this feature looks to be implemented.

I will write about the gRPC Response the the other git-issue