Configuring a Buffer Provider

[seanmonstar]

hyper currently manages its own read buffer to pull bytes from the IO transport. It tries to be an optimal default. But at times, people want finer-grained control. In order to be Flexible, we could consider how a user could provide a custom buffer management scheme.

The current behavior could become the default provider (which allows setting a max, or using adaptive growth). We could add new useful ones in hyper-util to begin with.

Some things to keep in mind:

• Adding a new generic to the existing Connection types isn't easy. It would require having a default set, which is fine, but anything already written expecting that generic to not exist could never allow a new provider.
• At the other end, Bytes must come out of the body::Incoming.
• Assuming that this ends in a new trait, it should consider backwards-compatibility hurdles that may come up in the future.
• It should be possible to implement a provider to share a pool, such that a whole bunch of connections can be sitting idle waiting for a read on potentially just a single buffer which is only used by the first connection to get more data. This would reduce the need for every idle connection to hold onto a chunk of memory waiting for a read wakeup.
• While the trait would be public API of hyper, internal unstable details could allow us to translate that for h2/h3, such that it could be used there too. (Such as feat: add recv buffer size option with improved recv buffer capacity recovery h2#904).

For implementation, some options could be:

• Something like Timer, where it's just boxed internally. The downside is that it would force all implementations to be Send + Sync.
• Perhaps it could be something added to rt::Read? Read::buf_provider() -> impl BufProvider?

After some discussion, if HIPs are approved, this is likely big enough of a feature to write up a HIP.