Different buffer pool implementations suitable for server use?

[tinkerware]

Hi,

Would you be willing to accept changes for a pluggable segment pool to make Okio more suitable for use in server environments with lots of concurrent buffer use?

[swankjesse]

Very interesting. Have you observed concurrency problems or is this just suspected? I'd love to do something where Okio detects its environment and acts accordingly.

[tinkerware]

Suspected from my first look, but I'm fairly certain the synchronisation between take and recycle will be heavily contended; the segmented approach to allocation benefits Buffer nicely but it means the pool is all but sure to get hammered as buffers are written to and read from; I notice that reads do not waste time in recycling segments.

You could probably amortize this to some extent, especially for larger buffers, by collecting segments to recycle and batch recycling them, for example after the main loop in Buffer.writeTo, but that really makes the performance more unpredictable for different workloads. On top of that, it would be great to experiment with thread local allocation and adaptive segment sizing similar to what Netty does.

[swankjesse]

I'm reluctant to add new APIs just for experimenting.

BUT...

I'd love to see some experimentation with it! Interested in seeing where the contention crops up in practice? Perhaps with some experimentation we can improve the default policy, and possibly also come up with a policy that works well both on Android and on the server.

I'm also interested to see what the sweet spot in segment sizes is. Do 16KiB segments justify the extra wasted memory in additional throughput? If you'd like to try this out, that would be most excellent.

[JakeWharton]

A potential path for this would be a sibling module that promotes package-private APIs to public, with an understanding that there is zero API stability or guarantees between releases. That would allow experimenting without having to build your own from source.

[tinkerware]

I'm reluctant to add new APIs just for experimenting.

I hear you. I should phrase this better; being able to experiment only precedes the ability to extend the library to take into account findings from those experiments. I wanted to make sure that being useful in server workloads is something you are interested in seeing.

Let me try to get a clearer picture of any current bottlenecks by cooking up a couple of JMH benchmarks, then we can talk specifics.

[swankjesse]

Though I haven't done much on this lately, I think we're going to need it. Here's more benchmarks, this time from Guava. google/guava#2112

[swankjesse]

Fixed recently.
#717