Zero-copy NIO channel read & write

[swankjesse]

I’m experimenting using okio.Buffer with SocketChannel.

One challenge is SocketChannel doesn’t offer APIs that fit Okio:

public class SocketChannel ... {
    ...
    public int read(ByteBuffer dst);
    public long read(ByteBuffer[] dsts, int offset, int length);
    public long read(ByteBuffer[] dsts);

    public int write(ByteBuffer src);
    public long write(ByteBuffer[] srcs, int offset, int length);
    public long write(ByteBuffer[] srcs);
}

I can accomplish my goals by creating a temporary ByteBuffer, then copying its results into an Okio Buffer with these existing functions from the ByteChannel supertype:

class Buffer {
  ...
  fun read(sink: ByteBuffer): Int
  fun write(source: ByteBuffer): Int
}

But copying an extra time sucks, let’s not do that!

A better alternative is @bnorm’s awesome ByteChannelSink sample, which uses ByteBuffer.wrap().

I’d like to get something like that into Okio, as extension functions on the appropriate NIO channel types:

  /**
   * Reads up to [byteCount] bytes from this into [sink].
   * 
   * This will read fewer bytes if this channel is exhausted.
   * 
   * It will also read fewer bytes if this channel is in non-blocking mode, and
   * reading more bytes would require blocking.
   * 
   * @return the number of bytes read, possibly 0. Returns -1 if zero bytes were
   *     read and this channel is exhausted.
   */
  fun ReadableByteChannel.read(sink: Buffer, byteCount: Long): Long

  /**
   * Writes up to [byteCount] bytes from [source] into into this.
   * 
   * This will write fewer bytes if this channel is ready to accept new data.  This
   * typically occurs when this channel is in blocking mode and writing more
   * bytes would require blocking.
   * 
   * @return the number of bytes written, possibly 0.
   */
  fun WritableByteChannel.write(source: Buffer, byteCount: Long = source.size): Long

Cache ByteBuffer instances?

Should we cache result of ByteBuffer.wrap() as a field on Segment ? It has the potential to be a frequent allocation, though it’s also one that the VM should be able to escape-analysis away. Looking at JOL, we could add a ByteBuffer field without immediately harming the size of Segment.

 ~/Development/jol (master) $ java -cp jol-samples/target/jol-samples.jar org.openjdk.jol.samples.JOLSample_01_Basic
 OFF  SZ         TYPE DESCRIPTION               VALUE
   0   8              (object header: mark)     N/A
   8   4              (object header: class)    N/A
  12   4          int Segment.pos               N/A
  16   4          int Segment.limit             N/A
  20   1      boolean Segment.shared            N/A
  21   1      boolean Segment.owner             N/A
  22   2              (alignment/padding gap)
  24   4       byte[] Segment.data              N/A
  28   4      Segment Segment.next              N/A
  32   4      Segment Segment.prev              N/A
- 36   4              (object alignment gap)
+ 36   4   ByteBuffer Segment.byteBuffer        N/A
 Instance size: 40 bytes
-Space losses: 2 bytes internal + 4 bytes external = 6 bytes total
+Space losses: 2 bytes internal + 0 bytes external = 2 bytes total

For now I’d like to start by not caching, especially since doing so would require Segment to be split into JVM and non-JVM declarations.

[yschimke]

I've been hoping okio would have some socket like interface for a while. Would make things like OkHttp/Wire working multiplatform possible. Possible implementations, 1) Socket, 2) TLS Socket, ...

Could you gain anything by expressing this problem with an Okio socket like interface?

[swankjesse]

Yep, I think that’s appropriate. Particularly for Kotlin/Native, where we don’t yet have a multiplatform socket.

[pull-vert]

Hi,
Java's NIO channels, and also IO sockets since Java 13 with the new default implementation NioSocketImpl (see Reimplementation of the Legacy Socket API) only read and write thanks to native (direct) ByteBuffers.

Java uses a direct ByteBuffer pool to avoid allocating and zero-fill a new direct ByteBuffer every time, like Okio does for segments, but it is always copying from heap to native memory on each operation.

Wrapping a byte[] in a heap-based ByteBuffer will not allow to avoid this memory copying operation from heap to native ByteBuffer, so it cannot be called zero-copy in my opinion. But it is indeed better than manually creating a new direct ByteBuffer because Java will use its own pool.

See IoUtil.java for NIO channels

static int write(FileDescriptor fd, ByteBuffer src, long position,
  boolean directIO, boolean async, int alignment,
  NativeDispatcher nd) throws IOException {
  if (src instanceof DirectBuffer) {
    return writeFromNativeBuffer(fd, src, position, directIO, async, alignment, nd);
  }

  // Substitute a native buffer
  ByteBuffer bb = Util.getTemporaryDirectBuffer(rem);
  try {
    bb.put(src);
   // ...

And NioSocketImpl for IO socket

private int tryWrite(FileDescriptor fd, byte[] b, int off, int len) throws IOException {
  ByteBuffer src = Util.getTemporaryDirectBuffer(len);
  try {
    src.put(b, off, len);
    return nd.write(fd, ((DirectBuffer)src).address(), len);
  } finally {
    Util.offerFirstTemporaryDirectBuffer(src);
  }
}