net/http: excessive mutex contention

[dvyukov]

The attached contention profile (collected with http://golang.org/cl/6443115 on
a parallel http load test) shows great amount of contention on
http.Transport.putIdleConn(), and the load test fails to scale beyond 4 cores.
I think we need try to reduce critical sections and see how it affects
performance/scalability.

Attachments:

1. http_contention.svg (48386 bytes)

[remyoudompheng]

Comment 1:

connectMethod.String() looks like a particularly heavy operation (calling URL.String and
strings.Join), could the attached patch be enough to solve the problem ?

Attachments:

1. net_http_transport.patch (1207 bytes)

[dvyukov]

Comment 2:

http://golang.org/cl/6454142
I take your patch as a base, and added some more optimizations. I did not measure them
separately, but the resulting speedups look quite impressive (-40%).

[dvyukov]

Comment 3:

New contention profile after applying http://golang.org/cl/6454142

Attachments:

1. cprof2.svg (63673 bytes)

[robpike]

Comment 4:

Labels changed: added priority-soon, removed priority-triage.

[dvyukov]

Comment 5:

This issue was closed by revision 20f6a8f.

Status changed to Fixed.