transport/http: update default transport settings

[tritone]

Updates the default HTTP transport to use a larger value for
MaxIdleConnsPerHost. This improves performance under high load
for the GCS client.

Copied from gerrit CL https://code-review.googlesource.com/c/google-api-go-client/+/55470

[broady]

This actually has another effect: the transport (http.DefaultTransport) would now no longer be shared between clients.

I don't know if that matters, but if it does, we would want the tweaked transport stored as a package-level variable rather than being copied every time a new client is created.

Otherwise, this looks good. I would tweak the build tags a bit:

//====dial.go===

// defaultBaseTransport returns the base HTTP transport.
// On App Engine, this is urlfetch.Transport.
// Otherwise, use a default transport, taking most defaults from
// http.DefaultTransport.
// If TLSCertificate is available, set TLSClientConfig as well.
func defaultBaseTransport(ctx context.Context, clientCertSource cert.Source) http.RoundTripper {
	if appengineUrlfetchHook != nil {
		return appengineUrlfetchHook(ctx)
	}

	trans := clonedTransport(http.DefaultTransport)
	if trans == nil {
		trans = fallbackBaseTransport()
	}
	trans.MaxIdleConnsPerHost = 100

	if clientCertSource != nil {
		trans.TLSClientConfig = &tls.Config{
			GetClientCertificate: clientCertSource,
		}
	}

	return trans
}

// fallbackBaseTransport is used in <go1.13 as well as in the rare case if http.DefaultTransport has been reassigned something that's not a *http.Transport.
func fallbackBaseTransport() *http.Transport {
	return &http.Transport{
		Proxy: http.ProxyFromEnvironment,
		DialContext: (&net.Dialer{
			Timeout:   30 * time.Second,
			KeepAlive: 30 * time.Second,
			DualStack: true,
		}).DialContext,
		MaxIdleConns:          100,
		MaxIdleConnsPerHost:   100,
		IdleConnTimeout:       90 * time.Second,
		TLSHandshakeTimeout:   10 * time.Second,
		ExpectContinueTimeout: 1 * time.Second,
	}
}


//====default_transport_go113.go===

// clonedTransport returns the given RoundTripper as a cloned *http.Transport.
// It returns nil if the RoundTripper can't be cloned or coerced to *http.Transport.
func clonedTransport(rt http.RoundTripper) *http.Transport {
	t, ok := rt.(*http.Transport)
	if !ok {
		return nil
	}
	return t.Clone()
}


//====default_transport_not_go113.go===

func clonedTransport(rt http.RoundTripper) *http.Transport { return nil }

oops

[tritone]

Thanks for your comments, @broady! I fixed the cloning and build tag factoring per your suggestion.

About the fact that the default transport is no longer shared between clients-- do you think I should address that in this PR? Let me know what you think.

[broady]

About the fact that the default transport is no longer shared between clients-- do you think I should address that in this PR? Let me know what you think.

Yeah, probably worth doing. Sent some code suggestion.