Implement flush coalescing in OkHttp

[bubenheimer]

OkHttpClientTransport currently forces the connection to use TCP_NODELAY: https://github.com/grpc/grpc-java/blob/master/okhttp/src/main/java/io/grpc/okhttp/OkHttpClientTransport.java#L426

For my use case (and, I would argue, other Android "next billion users" use cases) prioritizing low resource usage over performance seems to make more sense, so I would like to see this option to be configurable. I agree that enabling TCP_NODELAY seems the better default in general: #256

In my case I have a good number of low-throughput streaming RPCs that will stay open until the connection is closed. Speed and latency are secondary, low resource usage and reliability are important. Most of the streaming RPCs are primarily intended to let the server stream any changes in system state. For my Android "next billion users" use case I expect the connection to break fairly frequently, at which point the client will re-establish the streaming RPCs, so that it can catch up with the server, and the server can resume streaming any changes in system state. Consequently the first thing that will happen upon reconnect is all the streaming RPCs piping their header information down the new connection. With TCP_NODELAY enabled I expect this to mean several packets PER STREAMING RPC. Without that flag I would hope to see no more than a few packets TOTAL. Because the overall throughput of actual state data is low for this system and connections could break frequently, I think this could have a notable effect on total mobile data used.

I am still in the design phase for the system and identifying the potential bottlenecks from the perspective of a new gRPC & HTTP2 user, so please excuse and correct any false assumptions. I realize I could probably use just one streaming RPC to pipe data down to the client, but that kind of defeats the purpose of a protobuf-based RPC design, as I have different types of data to send. Being able to squeeze the connection initialization data into a few packets would make the overhead more negligible.

[ejona86]

To reduce the problem, we should do flush coalescing, probably in AsyncFrameWriter. That would improve performance for everyone. That's not been important up to this point because we generally don't expect very many concurrent RPCs on Android and so we've just optimized flushes per-RPC.

Streaming also causes additional flushes. The C implementation already added APIs to suppress the flushes when the application guarantees it'll notify grpc when it is done sending a batch of data. That would be on the table here as well, and benefit Netty as well. (Similar semantics to TCP_CORK.)

I have also heard reports that radio communications do a good job of combining packets at the physical layer, so TCP-level packet count may not directly relate to resource usage.

I'm not strongly against adding an option to avoid TCP_NODELAY, but I think I'd like to see that flush coalescing isn't enough. I do understand the delay TCP_NODELAY adds can prove quite helpful when latency is unimportant. But coalescing should be easy to implement, would work out-of-the-box, and avoids a Channel-level configuration for an RPC-level decision (whether the RPC is latency-sensitive). We'd really only need AsyncFrameWriter.flush() to cancel the previously queued flush, if there is one. That's easier than in Netty because okio already auto-flushes after 8KB is queued.