Size http2 buffers to allow concurrent streams

[liggitt]

http/2 requests from a given client multiplex over a single connection via streams, chopped up into frames.

The amount of data the client is allowed to send for a given stream and for the overall connection before acknowledgement is determined by the server's MaxUploadBufferPerStream and MaxUploadBufferPerConnection settings respectively, both defaulting to 1MB.

The number of concurrent streams a client is allowed to send over a single connection is determined by the server's MaxConcurrentStreams setting, defaulting to 250.

We observed a starvation issue with the kube aggregator's proxy client if handling of a POST through the aggregator to a backend server exceeded the 1MB buffer size AND the backend server required a second POST request through the aggregator to be handled before it could drain the first request's body.

Logically, if concurrent streams are allowed in a single connection, the connection buffer should be MaxUploadBufferPerStream*MaxConcurrentStreams to allow individual streams to make progress even when one stream is blocked.

This PR shrinks the MaxUploadBufferPerStream size to 256kb (which is still large enough to allow all the resources we saw in our test clusters to be sent in a single frame), and grows the MaxUploadBufferPerConnection to accomodate concurrent streams.

I'm also opening a golang issue, reproducer, and fix for the defaults for this

adjusted http/2 buffer sizes for apiservers to prevent starvation issues between concurrent streams

[liggitt]

/sig scalability
/sig api-machinery

[liggitt]

@kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-scalability-pr-reviews

/cc @deads2k

[wenjiaswe]

cc @lavalamp @yliaog

[wenjiaswe]

cc @jpbetz

[wojtek-t]

Thanks @liggitt !

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• staging/src/k8s.io/apiserver/OWNERS [liggitt,wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 67694, 64973, 67902). If you want to cherry-pick this change to another branch, please follow the instructions here.

[mbohlool]

does it make sense to add a test for this?

[liggitt]

I have a reproducer for a golang issue queued up, but exercising this bug requires an aggregated API server whose rest handler makes a call to an aggregated API… pretty involved for a bug I'd rather see fixed by the runtime defaults