net/http: make ResponseWriter buffer size configurable

[alberts]

What steps reproduce the problem?

I've been doing some tests here with a net/http server where the handler makes many
relatively small writes of binary data to the ResponseWriter. I am ranging over some
key-values with a LevelDB iterator, and writing a bunch of the values.

The underlying bufio.Writer's size is hard-coded at 4k:

http://golang.org/src/pkg/net/http/server.go#L433

With this size, my throughput is about 65 MB/sec. Increasing to 64k gets me to 90MB/sec
and a 1M buffer gets me to 140 MB/sec.

The throughput of writing net/http responses can be significantly improved by making the
write buffer configurable.

Discussion here:

https://groups.google.com/d/msg/golang-dev/OuFtcKEyGrg/VyKo_LqOhK8J

Please provide any additional information below.

The fix is two lines.

Given that there's still 173 Go 1.3 bugs open, I would implore you to let me send a CL
now rather than wait another 6 months. But anyway.

[gopherbot]

Comment 1 by islandberry33:

Because bufio Writer handles large writes without buffering
(https://code.google.com/p/go/source/browse/src/pkg/bufio/bufio.go#532), wrapping the
response writer with a large bufio Writer should achieve OP's goal.

[ianlancetaylor]

Comment 2:

Labels changed: added repo-main, release-go1.4.

[rsc]

Comment 3:

I agree with comment #1: put your own buffering in the handlers that need it, and
everything will work fine.

Status changed to WorkingAsIntended.

[bradfitz]

Comment 4:

I thought I looked into this and remembered it being outside the user's control. I
should've written down some notes at the time.