Native transport support. Off-heap byte buffers & pooling.

[MaximBreitman]

1. Use of native transport (Epoll/Kqueue if available, else NIO) would improve performance for the respective platforms.

2. Use & pool off-heap direct buffers would allow better memory control without creating unnecessary garbage for the GC.
   Obviously, (de-)serialization on the API side should also be switched to direct buffers to avoid moving to heap arrays.

The point is that all of this has long been implemented in the fairly user-friendly library Netty. So the question is, why not use Netty?
I read an earlier issue about this, but still. If there is a way to improve performance, why not do it?

Of course, you can use Java NIO and your own object pools (still better than using heap byte arrays), but Netty allows you to create even less garbage at the expense of natives.

#373

[scottf]

1. Use of NIO or netty would require a major overhaul / major version. It's under discussion, including using vertx.
2. I have read multiple discussions about byte[] versus ByteBuffer (which is what I'm assuming you mean by direct buffers). We have ByteArrayBuilder which uses ByteBuffer.

At one time I implemented ByteArrayBuilder with both byte[] and ByteBufer. Testing did not demonstrate any real performance difference, and we stuck with ByteBuffer under the covers just because of this type of conversation.
There are certainly places to tune this client to improve performance, maybve even more places where ByteArrayBuilder could be used, but otherwise it would require major overhaul.

[MaximBreitman]

I took a look at ByteArrayBuilder, and no, it's not a direct buffer, but a heap buffer, a wrapper over the heap byte array with a more convenient API. Using this wrapper itself takes heap memory, in addition to the byte array itself, so the consumption will probably be a bit more.

#allocate creates a HeapByteBuffer and #allocateDirect creates a DirectByteBuffer.

Replacing the allocate method with allocateDirect will not improve the situation. You need to keep track of allocated resources outside the heap; and the direct buffers themselves are not much faster than heap buffers, as far as I know, somewhere since Java 17, although I could be wrong. The idea is to take out the limited circulating resources so you don't have to take memory from the heap and resort to garbage collection unnecessarily.

Yes, these changes are critical. You'll have to replace not only the implementation, but also the API. It is unlikely to be much better if you end up converting bytes from off-heap to heap for a byte array. The sensible thing to do, it seems to me, would be to provide a Direct ByteBuffer to the end user (and return to pool; the request itself should preferably be processed in another thread, e.g. in a virtual thread). This could probably be covered with additional wrappers for more simplicity; maybe take a look at the Foreign Function Memory API

[MaximBreitman]

Should it be transferred to the issues section?