Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: ResponseHeaderTimeout can cause the Heap & GC to grow/increase rapidly #9104

Closed
fraenkel opened this issue Nov 14, 2014 · 5 comments

Comments

Projects
None yet
5 participants
@fraenkel
Copy link
Contributor

commented Nov 14, 2014

go1.3.3 although go1.4 will have this problem as well

When using the DefaultTransport, in our case it was via the ReverseProxy, the
ResponseHeaderTimeout will create a timer that is never closed.  The code relies on
expiring.
If you have a proxy under load and a timeout that is long in our case 15 minutes but
even anything in the range of 2-5 minutes, the number of timers starts to overwhelm the
Heap and GC.

In our simple runs under load, we can generate 10k req/s.  We see the heap grow from 2Mb
to 90Mb all due to timers.  The GC cycle starts at 2ms and grows to 120ms.

Our current workaround is to avoid setting the ResponseHeaderTimeout and rely on
DeadlineTimeouts.  Closing the timers created persistConn.roundTrip() would also
alleviated the explosion.
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Nov 14, 2014

Comment 1:

Labels changed: added repo-main, release-go1.5.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Nov 14, 2014

Comment 2:

Nice find, thanks! I'll fix after Go 1.4 is out.

Owner changed to @bradfitz.

Status changed to Accepted.

@dvyukov

This comment has been minimized.

Copy link
Member

commented Nov 14, 2014

Comment 3:

Is it a non-stopped timer?
@bradfitz

This comment has been minimized.

Copy link
Member

commented Nov 14, 2014

Comment 4:

var respHeaderTimer <-chan time.Time
...
                        if d := pc.t.ResponseHeaderTimeout; d > 0 {
                                respHeaderTimer = time.After(d)
                        }
@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 13, 2014

@bradfitz bradfitz closed this in 0e4ee0c Dec 13, 2014

@bradfitz bradfitz modified the milestone: Go1.5 Dec 16, 2014

@bradfitz bradfitz removed the release-go1.5 label Dec 16, 2014

@golang golang locked and limited conversation to collaborators Jun 25, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.