net/http: impossible to use both CloseNotify and Hijack in the same Server

[bgentry]

The existing implementation makes it impossible to run a Server with keep-alive that both gets notified about disconnecting clients and is capable of handling upgrade requests (such as to websocket protocol).

For example, consider the recent change to utilize CloseNotifier within the httputil.ReverseProxy: ececbe8

That's an important change for a reverse proxy because it allows upstream servers to stop doing work for clients that have gone away. Without that behavior, servers are much more vulnerable to resource exhaustion by misbehaving clients that disconnect immediately after making a request.

However, suppose that a Server not only wanted to defend against disconnecting clients, but also wanted to be able to handle HTTP Upgrade (or websocket) requests. That can only be implemented in http.Server by using the ResponseWriter's Hijacker interface to hijack the underlying net.Conn.

However, those two features are explicitly blocked from being used together. There's a good underlying reason for that: once CloseNotify() is called, there's an io.Copy reading from the underlying net.Conn that cannot be cleanly interrupted.

I think it's important to be able to use both of these features together in the same Server. One could imagine adding websocket or more general HTTP Upgrade support to the ReverseProxy, or simply supporting such features from a Server that also wants to know when clients go away on normal requests.

What version of Go are you using (go version)?

go1.4.1

What operating system and processor architecture are you using?

Darwin amd64

What did you do?

I attempted to Hijack() on a re-used server connection, one that had been setup for CloseNotify() on a previous request.

What did you expect to see?

I expected to be able to hijack the connection, even though in a previous request I had asked to be notified about disconnecting clients.

What did you see instead?

http: Hijack is incompatible with use of CloseNotifier

[bgentry]

I made an attempt to solve this in https://go-review.googlesource.com/#/c/3821/

This change allows connections that have been used with CloseNotify to
also be used with Hijack. It does so by returning a wrapped net.Conn
type that reads from the closeNotify pipe reader, rather than directly
from the net.Conn.

[benburkert]

CloseNotify checks the state of the underlying net.Conn of a conn, so another approach is to check if the underlying net.Conn is itself a CloseNotifier, and if so return the result of calling CloseNotify(). In this case it should be safe to hijack the conn after a CloseNotify because there was no io.Copy goroutine issuing reads to the net.Conn directly.

The necessary change should be quite small and overhead negligible: benburkert@3c2fd2f

[bgentry]

I do like the idea of relying on a CloseNotifier on the net.Conn. Seems like that could be useful elsewhere.

To do that, you'd have to either add CloseNotify() to the built-in net.Conn types like net.TCPConn, or use a custom net.Listener that returns a wrapped net.Conn type which fulfills CloseNotifier.

[gopherbot]

CL https://golang.org/cl/3821 mentions this issue.

[pwaller]

Just to note that this is causing problems for Docker, too, so it would be really great to get this fixed for 1.5. moby/moby#12845

[rsc]

I apologize, but I don't believe we'll be able to fix this in time for Go 1.5.

[bgentry]

Any updates on whether there's an approach that would work for this? I would work on my CL further if it's actually an acceptable solution.

[bmizerany]

Are you trying to hijack after closenotify? I.e.

• do long running http land stuff, maybe hijack?
• yes: hijack and start doing custom stuff
  On Wed, Oct 28, 2015 at 6:51 PM Blake Gentry notifications@github.com
  wrote:

Any updates on whether there's an approach that would work for this? I
would work on my CL further if it's actually an acceptable solution.

—
Reply to this email directly or view it on GitHub
#9763 (comment).

[bgentry]

Are you trying to hijack after closenotify?

@bmizerany I think the situation is pretty well described in the original issue. You can't hijack a connection that previously had a CloseNotifier set up on it (even if it was set up on a previous request). Essentially this means that you can't reliably use both CloseNotify and Hijack in the same http.Server (sharing the same client connection pool).

CloseNotify is essential to a production server (protecting it from continuing to work when the client goes away). At the same time, Hijack is required for Websocket or just general HTTP Upgrade support. As a result, it's impossible to run an http.Server that has both production-ready support for regular HTTP requests, and also support for Websockets/HTTP upgrade.

Please read the original issue and let me know if anything is poorly explained ✌️

[bmizerany]

I saw this in email and mistook it for a forum post, not Github, so I didn't get the back story. I was going to suggest building your own CloseNotifierConn, but it looks like you already did. Nevermind.

[gopherbot]

CL https://golang.org/cl/17750 mentions this issue.

[bradfitz]

@bgentry, check out the latest version of https://golang.org/cl/17750 (patchset 2+), which has a fix for this bug. Unlike your change, it doesn't returned a custom net.Conn type, so users can still type-assert it to a *net.TCPConn or *tls.Conn, if that matters. Instead of a background io.Copy always running now, we only read a single byte in the background.

[gopherbot]

CL https://golang.org/cl/31173 mentions this issue.