Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add http/2 cleartext server (h2c) support #3227

Closed
Zetanova opened this issue Apr 4, 2020 · 8 comments · Fixed by #3289
Closed

Add http/2 cleartext server (h2c) support #3227

Zetanova opened this issue Apr 4, 2020 · 8 comments · Fixed by #3289
Labels
feature
Milestone

Comments

@Zetanova
Copy link

@Zetanova Zetanova commented Apr 4, 2020

The normal case is always to encrypt http2 network connections,
but since there are cases in gRPC and service-meshes where the upstream
is over an unix domain stocket (UDS), http/2 cleartext (h2c) would be required.

In a TLS connection the client can negate the h2 protocol with the help of TLS-ALPN header.

But in the case of http/2 cleartext (h2c) there are two different methods:

  1. h2c upgrade
  2. h2c prior knowledge

The golang native server can negate it already by inserting the h2c middleware Handler from golang.org/x/net/http2/h2c with h2c.NewHandler(handler, h2s)
h2c-golang-example

By simply extending the listener factory caddy2 could then serve
h2c connections by h2c update and also at the same time by h2c prior knowledge

I think, the h2c-handler can be added here:

// set up each listener modifier
if srv.ListenerWrappersRaw != nil {
vals, err := ctx.LoadModule(srv, "ListenerWrappersRaw")
if err != nil {
return fmt.Errorf("loading listener wrapper modules: %v", err)
}
var hasTLSPlaceholder bool
for i, val := range vals.([]interface{}) {
if _, ok := val.(*tlsPlaceholderWrapper); ok {
if i == 0 {
// putting the tls placeholder wrapper first is nonsensical because
// that is the default, implicit setting: without it, all wrappers
// will go after the TLS listener anyway
return fmt.Errorf("it is unnecessary to specify the TLS listener wrapper in the first position because that is the default")
}
if hasTLSPlaceholder {
return fmt.Errorf("TLS listener wrapper can only be specified once")
}
hasTLSPlaceholder = true
}
srv.listenerWrappers = append(srv.listenerWrappers, val.(caddy.ListenerWrapper))
}
// if any wrappers were configured but the TLS placeholder wrapper is
// absent, prepend it so all defined wrappers come after the TLS
// handshake; this simplifies logic when starting the server, since we
// can simply assume the TLS placeholder will always be there
if !hasTLSPlaceholder && len(srv.listenerWrappers) > 0 {
srv.listenerWrappers = append([]caddy.ListenerWrapper{new(tlsPlaceholderWrapper)}, srv.listenerWrappers...)
}
}

If required http1 coud be disabled to accept only h2c-prior-knowledge,
but then caddy would need to bind the http2 server directly to the tcp-connection like

l, err := net.Listen("tcp", "0.0.0.0:1010")
checkErr(err, "while listening")

fmt.Printf("Listening [0.0.0.0:1010]...\n")
for {
    conn, err := l.Accept()
    checkErr(err, "during accept")

    server.ServeConn(conn, &http2.ServeConnOpts{
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintf(w, "Hello, %v, http: %v", r.URL.Path, r.TLS == nil)
        }),
    })
}

Because the h2c-handler covers both options, this feature could also be omitted.

Maybe the h2c handler can be added by extending the
JSON Config Structure › apps › http › servers › listener_wrappers

Demo Caddyfile for UDS

http://

bind unix//tmp/caddy.sock

reverse_proxy {
    to unix//temp/upstream.sock
    transport http {
        versions h2c
    }
}

Demo Caddyfile for TCP

http://

bind 127.0.0.1

reverse_proxy {
    to http://127.0.0.1:2012
    transport http {
        versions h2c
    }
}
@francislavoie francislavoie added the feature label Apr 4, 2020
@Zetanova Zetanova changed the title Add h2c bind support Add http/2 cleartext server (h2c) support Apr 4, 2020
@mholt
Copy link
Member

@mholt mholt commented Apr 6, 2020

To clarify:

service-meshes where the upstream is over an unix domain stocket (UDS), http/2 cleartext (h2c) would be required.

The socket used (UDS or otherwise) has nothing to do with whether TLS is used. It's a totally separate thing.

@Zetanova Also, I don't believe we need to use the method with server.ServeConn() nor mess with listeners at all, since the h2c.NewHandler() method supports both upgrade and prior knowledge.

@Zetanova
Copy link
Author

@Zetanova Zetanova commented Apr 6, 2020

@mholt yes, the method with server.ServeConn()should not be required.
Most UDS stream setups are unencrypted.

mholt added a commit that referenced this issue Apr 6, 2020
@mholt
Copy link
Member

@mholt mholt commented Apr 6, 2020

@Zetanova Alrighty, I have pushed experimental support for h2c server in the h2c branch:
b61e1a2

It is enabled by setting "insecure_h2c": true in the server struct (so, for example, adjacent to "listen"). (Sorry, didn't have time to whip up Caddyfile support yet.)

As before, I haven't tested it; will you try it out and see how it goes?

@Zetanova
Copy link
Author

@Zetanova Zetanova commented Apr 6, 2020

@mholt thx a lot!

looks good, the listener is working.

But the gRPC-response error/bug from the other new "h2c upstream" feature still breaks the gRPC communication. I hope that it will work with an upstream over normal tcp-h2

Test setup:
[gloang gRPC client] <- uds-h2c -> [caddy] <- tcp-h2c -> [kestrel gRPC server]

@mholt
Copy link
Member

@mholt mholt commented Apr 6, 2020

@Zetanova

But the gRPC-response error/bug from the other new "h2c upstream" feature still breaks the gRPC communication.

Do you mean this one?

@Zetanova
Copy link
Author

@Zetanova Zetanova commented Apr 6, 2020

@mholt No, this one

the gRPC Response in caddy gets somehow to transfered correctly back to the gRPC-client
Behavior the same with [golang gRPC client] <h2c> [caddy] <h2c> [dotnet gRPC server] and [dotnet gRPC client] <h2|h2c> [caddy] <h2c> [dotnet gRPC server]

i will test now [dotnet gRPC client] <h2|h2c> [caddy] <h2> [dotnet gRPC server]

@mholt
Copy link
Member

@mholt mholt commented Apr 6, 2020

Gotcha...

I'm afraid I don't know enough about gRPC to debug that one :( I'll need some hints to make much more progress. (Hopefully it is simple!)

@Zetanova
Copy link
Author

@Zetanova Zetanova commented Apr 6, 2020

@mholt gRPC info both gRPC clients (dotnet and golang) complaining about a missing response status code

Listening with h2c looks like to work now!
So this feature looks to be implemented.

I will write about the gRPC Response the the other git-issue

@mholt mholt added this to the 2.x milestone Apr 6, 2020
@mholt mholt removed this from the 2.x milestone Apr 14, 2020
@mholt mholt added this to the 2.1 milestone Apr 14, 2020
@mholt mholt linked a pull request Apr 21, 2020 that will close this issue
mholt added a commit that referenced this issue May 5, 2020
* reverse_proxy: Initial attempt at H2C transport/client support (#3218)

I have not tested this yet

* Experimentally enabling H2C server support (closes #3227)

See also #3218

I have not tested this

* reverseproxy: Clean up H2C transport a bit

* caddyhttp: Update godoc for h2c server; clarify experimental status

* caddyhttp: Fix trailers when recording responses (fixes #3236)

* caddyhttp: Tweak h2c config settings and docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants