net/http: add http1 and http2 Server IdleTimeout #14204

bradfitz · 2016-02-02T23:53:22Z

Add http.Server.IdleTimeout to limit the keep-alive idle time for http1 and http2 servers.

gopherbot · 2016-02-03T01:00:15Z

CL https://golang.org/cl/19159 mentions this issue.

Updates golang/go#14204 Change-Id: Id2598c77e2677a50988c00adc8751a9b87751202 Reviewed-on: https://go-review.googlesource.com/19159 Reviewed-by: Andrew Gerrand <adg@golang.org>

odeke-em · 2016-04-04T07:50:07Z

If I may ask, this is going to entail us adding MaxIdleTimeout and MaxActiveTimeout to Server, right? And also we'll add defaults as you in have in CL 19159, true?
Or should we just expose a new function IdleTimeoutHook so that it takes these durations which if <= 0 means we should use the default durations?

gopherbot · 2016-04-04T09:00:48Z

CL https://golang.org/cl/21491 mentions this issue.

bradfitz · 2016-09-01T23:31:17Z

@odeke-em, the naming here is much easier if we don't consider HTTP/2 and PING frames as activity. Then we'd just call it IdleTimeout and be done with it.

But with HTTP/2, do we care about clients sending a PING or other useless frame every N seconds vs. actually sending real requests every N seconds? If so, we need names for both types of idle (network idle vs no-HTTP-requests idle). Or we define IdleTimeout to mean one thing, and give ReadTimeout the other meaning (see #16958 (comment))?

Opinions welcome. Ideally with sample documentation you'd tell users.

I got as far as:

        // ReadTimeout optionally specifies a maximum duration before                                                                                        
        // timing out reads of request headers. It does not include                                                                                          
        // request bodies. For requests after the first on a                                                                                                 
        // connection, this is timer starts upon receiving the first                                                                                         
        // byte. For keep-alive duration, see IdleTimeout.                                                                                                   
        ReadTimeout  time.Duration

        // IdleTime, is, uh....                                                                                                                              
        IdleTimeout time.Duration

/cc @rhysh @broady @adg @bmizerany

bradfitz · 2016-09-01T23:31:52Z

/cc @prabrisat1 too.

freeformz · 2016-09-03T15:17:31Z

// IdleTimeout is the maximum time between requests that a connection 
can stay idle. The timeout starts as soon as a connection is started and 
after each request is finished. It is canceled when a request starts.

bradfitz · 2016-09-03T15:46:49Z

So PING frames don't count, then? That's a valid answer.

Now that I think about, we should disable the Transport's TCP keep-alive when HTTP/2 is negotiated, since PING frames and TCP keep-alive packets serve the same purpose of keeping the connection alive from NAT timeouts and whatnot.

freeformz · 2016-09-06T17:59:43Z

WRT request bodies vs request headers ... That does mean a malicious client could send headers fully and then bodies as slowly as possible. I guess though you would use a context with a timeout or the like to protect against that in the handler?

freeformz · 2016-09-06T18:04:54Z

That probably complicates code though? Should ReadTimeout be canceled after the request's Body is closed, not when the headers are finished being read?

bradfitz · 2016-09-06T18:12:21Z

WRT request bodies vs request headers ... That does mean a malicious client could send headers fully and then bodies as slowly as possible. I guess though you would use a context with a timeout or the like to protect against that in the handler?

Yes, that's the job of the handler or caller of http.Transport.RoundTrip to guard against. We only give a timeout for headers, since the user isn't in control there.

Should ReadTimeout be canceled after the request's Body is closed, not when the headers are finished being read?

SetReadDeadline is currently only called before a TLS handshake, and before reading request headers, and is never disabled.

So yes, to work similarly to Go 1.0-1.7, we should probably not reset it after the headers and have it apply to the whole request, even if that's slightly odd.

freeformz · 2016-09-06T18:59:45Z

Can the deadlines (since this is what is currently used) even be reset in the middle of a Read?

bradfitz · 2016-09-06T23:36:28Z

Why would we be in the middle of a read?

freeformz · 2016-09-07T00:21:10Z

@bradfitz Actually you wouldn't be AFAICT. Not sure why I thought that earlier (in an internal convo about this issue).

Fix a regression from Go 1.5 to Go 1.6: when we introduced automatic HTTP/2 support in Go 1.6, we never handled Server.ReadTimeout. If a user set ReadTimeout, the net/http package would set the read deadline on the connection during the TLS handshake, but then the http2 package would never disarm it, killing the likely-still-in-use connection after the timeout period. This CL changes it to disarm the timeout after the first request headers, similar to net/http.Server. Unlike net/http.Server, we don't re-arm it on each idle period, because the definition of idle is more complicated with HTTP/2. No tests here for now. Tests will be in the go repo once all the knobs are in place and this is re-bundled into std, testing both http1 and http2. Updates golang/go#16450 (minimal fix for now) Updates golang/go#14204 (considering a new http1+http2 IdleTimeout knob) Change-Id: Iaa1570c118efda7dc0a65ba84cd77885699ec8fc Reviewed-on: https://go-review.googlesource.com/30077 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>

gopherbot · 2016-10-22T19:50:41Z

CL https://golang.org/cl/31727 mentions this issue.

Also remove some stale TODOs and finish one of the TODOs, ignoring more incoming frames when the connection is in GOAWAY mode. Also, fix independently-broken transport test from a change in std: https://golang.org/cl/31595 Updates golang/go#14204 Change-Id: Iae8b02248464d613979c30d8f86eacb329cd262f Reviewed-on: https://go-review.googlesource.com/31727 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

gopherbot · 2016-10-25T23:50:44Z

CL https://golang.org/cl/32024 mentions this issue.

…meout Updates #14204 Updates #16450 Updates #16100 Change-Id: Ic283bcec008a8e0bfbcfd8531d30fffe71052531 Reviewed-on: https://go-review.googlesource.com/32024 Reviewed-by: Tom Bergan <tombergan@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>

gopherbot · 2016-10-27T20:10:40Z

CL https://golang.org/cl/32230 mentions this issue.

This makes ConfigureServer initialize the http2 Server's IdleTimeout from the http1 Server configuration, using the same rules as IdleTimeout in Go 1.8: first use IdleTimeout if specified, else fall back to the old ReadTimeout. Updates golang/go#14204 Change-Id: I4dee971f8416ef0cbf99335a199c46355f9ab09d Reviewed-on: https://go-review.googlesource.com/32230 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tom Bergan <tombergan@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

gopherbot · 2016-10-29T18:50:50Z

CL https://golang.org/cl/32322 mentions this issue.

gopherbot · 2016-10-29T19:05:31Z

CL https://golang.org/cl/32323 mentions this issue.

This is an alternate implementation of https://golang.org/cl/32230 which is nicer to the dead code elimination in Go's linker. The old implementation causes a test in the net/http package to fail: https://storage.googleapis.com/go-build-log/2c24cf88/linux-amd64_39728ac9.log ... since it caused the cmd/go binary to link in the whole http1 & http2 Server code, due to the init func and slice which referenced those symbols. Instead, use an explicit func. This aso includes the tests meant to be in CL 32230 but which I'd failed to git add earlier. Updates golang/go#14204 Change-Id: I13dc138bf0c4df65bc282133ee94036b7f84ab70 Reviewed-on: https://go-review.googlesource.com/32323 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Martin Möhrmann <martisch@uos.de>

bradfitz self-assigned this Feb 2, 2016

bradfitz added this to the Go1.7 milestone Feb 2, 2016

bradfitz mentioned this issue May 2, 2016

net/http: send and understand the hop-by-hop Keep-Alive header #15508

Open

tpng mentioned this issue May 9, 2016

Change in proxy config not reflected by in-process restart (http2 keep-alive connection not closed) caddyserver/caddy#809

Closed

bradfitz added the FeatureRequest label May 10, 2016

rsc modified the milestones: Go1.8, Go1.7 May 18, 2016

tpng mentioned this issue Jun 7, 2016

Close idle connections after graceful shutdown timeout caddyserver/caddy#870

Merged

bradfitz mentioned this issue Sep 1, 2016

net/http: ReadTimeout Breaks Server CloseNotify Functionality For Keep-Alive Connections #16958

Closed

bradfitz mentioned this issue Sep 30, 2016

net/http: http.Server.ReadTimeout never reset in http2 #16450

Closed

quentinmit added the NeedsFix label Oct 7, 2016

gopherbot closed this as completed in 4b130f9 Oct 29, 2016

golang locked and limited conversation to collaborators Oct 29, 2017

gopherbot added the FrozenDueToAge label Oct 29, 2017

rsc unassigned bradfitz Jun 23, 2022

net/http: add http1 and http2 Server IdleTimeout #14204

net/http: add http1 and http2 Server IdleTimeout #14204

bradfitz commented Feb 2, 2016

gopherbot commented Feb 3, 2016

odeke-em commented Apr 4, 2016

gopherbot commented Apr 4, 2016

bradfitz commented Sep 1, 2016

bradfitz commented Sep 1, 2016

freeformz commented Sep 3, 2016 •

edited

bradfitz commented Sep 3, 2016

freeformz commented Sep 6, 2016 •

edited

freeformz commented Sep 6, 2016

bradfitz commented Sep 6, 2016

freeformz commented Sep 6, 2016

bradfitz commented Sep 6, 2016

freeformz commented Sep 7, 2016

gopherbot commented Oct 22, 2016

gopherbot commented Oct 25, 2016

gopherbot commented Oct 27, 2016

gopherbot commented Oct 29, 2016

gopherbot commented Oct 29, 2016

net/http: add http1 and http2 Server IdleTimeout #14204

net/http: add http1 and http2 Server IdleTimeout #14204

Comments

bradfitz commented Feb 2, 2016

gopherbot commented Feb 3, 2016

odeke-em commented Apr 4, 2016

gopherbot commented Apr 4, 2016

bradfitz commented Sep 1, 2016

bradfitz commented Sep 1, 2016

freeformz commented Sep 3, 2016 • edited

bradfitz commented Sep 3, 2016

freeformz commented Sep 6, 2016 • edited

freeformz commented Sep 6, 2016

bradfitz commented Sep 6, 2016

freeformz commented Sep 6, 2016

bradfitz commented Sep 6, 2016

freeformz commented Sep 7, 2016

gopherbot commented Oct 22, 2016

gopherbot commented Oct 25, 2016

gopherbot commented Oct 27, 2016

gopherbot commented Oct 29, 2016

gopherbot commented Oct 29, 2016

freeformz commented Sep 3, 2016 •

edited

freeformz commented Sep 6, 2016 •

edited