net/http: implementing size capped connection pools

[brknstrngz]

Hi,

This issue came up a few times already on golang-nuts. We could use an upper bound on the HTTP connection pool (active + idle) size and either block the caller until a connection becomes available, or return an error.

One of the suggested approaches is to provide a custom Transport that does the bookkeeping inside Dial(). That works only partially, as after the HTTP Client call (Do() or any of the verb helpers) returns there is no way to get a reference to the underlying connection and return that to the pool.

net/httputil contains a ClientConn type that seems to have some potential, but it is considered old (and probably will disappear soon). Even so, as far as I can tell the only way to grab the underlying TCP connection is to hijack it, which in turn forces one to handle the entire HTTP logic themselves.

A naive approach would be to pool HTTP requests instead, but that is not really the desired approach.

Thanks for reading!

[methane]

I think database/sql.DB is nice pool. It can be custimized:

• Max conns
• Max idle conns
• Max lifetime

[mtojek]

@methane proposal seems to be fair. Are there any changes to see this feature introduced? I'm referring mainly to HTTP/1.1.

/cc @bradfitz