New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: reduce allocations in (Header).clone #29915

NWilson opened this Issue Jan 24, 2019 · 3 comments


None yet
5 participants
Copy link

NWilson commented Jan 24, 2019

What version of Go are you using (go version)?

go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?


What did you do?

When using net/http's server, and profiling with pprof shows that the "Header.clone" function takes up a lot of allocations (4% of my application's allocations, actually).


Could we improve the function to perform fewer allocations!

(NB. Note that Reader.ReadMIMEHeader in net/textproto already uses exactly this same trick, of using three-argument slicing to build the http.Header using a single []string. So the pattern already exists in the codebase.)

--- /tmp/header-v1.go   2019-01-24 11:18:01.182761218 +0000
+++ /tmp/header-v2.go   2019-01-24 11:17:42.367569268 +0000
@@ -1,9 +1,17 @@
 func (h Header) clone() Header {
  h2 := make(Header, len(h))
+ sliceLen := 0
+ for _, vv := range h {
+  sliceLen += len(vv)
+ }
+ slice := make([]string, sliceLen)
+ sliceLen = 0
  for k, vv := range h {
-  vv2 := make([]string, len(vv))
+  vv2 := slice[sliceLen:sliceLen+len(vv):sliceLen+len(vv)]
+  sliceLen += len(vv)
   copy(vv2, vv)
   h2[k] = vv2
  return h2

This comment has been minimized.

Copy link

mvdan commented Jan 24, 2019

I don't think there's a need to raise an issue for this. Assuming there already are benchmarks that would cover this optimisation, you can just send a CL with the change and the benchmark numbers showing the improvement.

@bcmills bcmills changed the title Reduce number of allocations in HTTP server net/http: reduce allocations in (Header).clone Jan 29, 2019


This comment has been minimized.

Copy link

bcmills commented Jan 29, 2019

CC @bradfitz for net/http.

Note that number of allocations doesn't necessarily correlate with overall throughput or latency. (Sometimes, collecting the allocation is less work for the program than avoiding it.)

In this case, the proposed change introduces a time/space tradeoff: if the user stores one particular slice, the may end up pinning the entire larger slice where before most of it could be collected.

(You'd want to support a change with benchmarks either way.)

@bcmills bcmills added the Performance label Jan 29, 2019

@bcmills bcmills added this to the Unplanned milestone Jan 29, 2019


This comment has been minimized.

Copy link

bradfitz commented Jan 29, 2019

Sounds fine to me. I was actually thinking about this recently when reviewing something else.

Go ahead and send a change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment