How to correctly reutilize the ByteBuf

[PedroMPagani]

Hey guys, currently I'm modifying a software in which is the Minecraft Server software, the main goal here is to achieve the reutilization of the ByteBuf of what we call Packets, Packets are basically lots of bytes used to communicate between client side and backend, my main goal is to avoid many costy things for the netty threads.

    protected void encode(ChannelHandlerContext channelHandlerContext, Packet<?> packet, ByteBuf byteBuf) throws Exception {
        ConnectionProtocol connectionProtocol = channelHandlerContext.channel().attr(Connection.ATTRIBUTE_PROTOCOL).get();
        if (connectionProtocol == null) {
            throw new RuntimeException("ConnectionProtocol unknown: " + packet);
        } else {
            Integer integer = connectionProtocol.getPacketId(this.flow, packet);
            if (LOGGER.isDebugEnabled()) {
                LOGGER.debug(Connection.PACKET_SENT_MARKER, "OUT: [{}:{}] {}", channelHandlerContext.channel().attr(Connection.ATTRIBUTE_PROTOCOL).get(), integer, packet.getClass().getName());
            }

            if (integer == null) {
                throw new IOException("Can't serialize unregistered packet");
            } else {
                /*FriendlyByteBuf org = packet.getSerialized(null);
                if(org != null){
                    ByteBuf b = org.retain();
                    Bukkit.broadcastMessage("Length: " + b.capacity());
                    Bukkit.broadcastMessage("Legnth.2" + byteBuf.capacity());
                    ByteBuffer original = b.nioBuffer();
                    ByteBuffer clone = byteBuf.nioBuffer();
                    original.rewind();//copy from the beginning
                    clone.put(original);
                    original.rewind();
                    clone.flip();
                    Bukkit.broadcastMessage("Deep clone packet.");
                    return;
                }*/

                FriendlyByteBuf friendlyByteBuf = new FriendlyByteBuf(byteBuf);
                friendlyByteBuf.writeVarInt(integer);
                friendlyByteBuf.adventure$locale = channelHandlerContext.channel().attr(PaperAdventure.LOCALE_ATTRIBUTE).get(); // Paper
                try {
                    int i = friendlyByteBuf.writerIndex();
                    packet.write(friendlyByteBuf);
                    int j = friendlyByteBuf.writerIndex() - i;
                    if (j > 8388608) {
                        throw new IllegalArgumentException("Packet too big (is " + j + ", should be less than 8388608): " + packet);
                    } else {
                        int k = channelHandlerContext.channel().attr(Connection.ATTRIBUTE_PROTOCOL).get().getId();
                        JvmProfiler.INSTANCE.onPacketSent(k, integer, channelHandlerContext.channel().remoteAddress(), j);
                    }
                } catch (Throwable var10) {
                    LOGGER.error("Packet encoding of packet ID {} threw (skippable? {})", integer, packet.isSkippable(), var10); // Paper - Give proper error message
                    if (packet.isSkippable()) {
                        throw new SkipPacketException(var10);
                    } else {
                        throw var10;
                    }
                }

                // Paper start
                int packetLength = friendlyByteBuf.readableBytes();
                if (packetLength > MAX_PACKET_SIZE) {
                    throw new PacketTooLargeException(packet, packetLength);
                }
                // Paper end
            }
        }
    }

This is the current code for the PacketEncoder class, which is the encoder MessageToByteEncoder.

As an example, there is a class called ClientboundSetEntityDataPacket in which it sends information about entities the user see, but that's very costy because it has lots of deep loops in it to serialize all the data, the main problem is, that packet can be sent a lot of times and every single time, even if using the same instance of ClientboundSetEntityDataPacket, it will have to re-serialize the data into a new ByteBuf, and go all over again in the deep loops.

public class ClientboundSetEntityDataPacket implements Packet<ClientGamePacketListener> {
    private final int id;
    @Nullable
    private final List<SynchedEntityData.DataItem<?>> packedItems;
    private @Nullable FriendlyByteBuf serializedByteBuf;

    public ClientboundSetEntityDataPacket(int id, SynchedEntityData tracker, boolean forceUpdateAll) {
        this.id = id;
        if (forceUpdateAll) {
            this.packedItems = tracker.getAll();
            tracker.clearDirty();
        } else {
            this.packedItems = tracker.packDirty();
        }

    }

    public ClientboundSetEntityDataPacket(FriendlyByteBuf buf) {
        this.id = buf.readVarInt();
        this.packedItems = SynchedEntityData.unpack(buf);
    }

    @Override
    public void write(FriendlyByteBuf buf) {
        if(serializedByteBuf == null){
            buf.writeVarInt(this.id);
            SynchedEntityData.pack(this.packedItems, buf);
            this.serializedByteBuf = buf;
        }
    }

    @Override
    public void handle(ClientGamePacketListener listener) {
        listener.handleSetEntityData(this);
    }

    @Nullable
    public List<SynchedEntityData.DataItem<?>> getUnpackedData() {
        return this.packedItems;
    }

    public int getId() {
        return this.id;
    }

    @org.jetbrains.annotations.Nullable
    @Override
    public FriendlyByteBuf getSerialized(FriendlyByteBuf target){
        if(this.serializedByteBuf != null){
            return this.serializedByteBuf;
        }
        return null;
    }

}

What I am trying to achieve instead is you can see I kept a reference of the FriendlyByteBuf class, it is basically an extension of the ByteBuf class, is to make it so if the packet gets serialized once, I would just re-use the bytes of the ByteBuf, and pass them to the ByteBuf reference in the encode method, that way I wouldn't have to go through these deep loops again and again, but the main issue is, every single time I try to re-utilize that packet(as you can see my commented code), I get an exception
io.netty.util.IllegalReferenceCountException: refCnt: 0

What am I doing wrong?

[chrisvest]

Every ChannelHandlerContext.write() will release() the buffers when the data is handed over to the kernel.

You need to add retain() calls to keep the buffers alive. The retain() and release() calls will pair up to cancel out, until a release() is paired with the buffer allocation itself, and the buffer is deallocated. The refCnt: 0 message is telling you that the buffer has been deallocated.

[PedroMPagani]

Where would I call the .retain()?

[chrisvest]

I don't see any reads or writers in the code you shared, but it would be before that.

[PedroMPagani]

After calling .retain() in a line of code, the error I get is another, but this one isn't really making sense to me:

                FriendlyByteBuf org = packet.getSerialized(null);
                if(org != null){
                    ByteBuffer clone = byteBuf.nioBuffer();
                    FriendlyByteBuf friendlyByteBuf = new FriendlyByteBuf(byteBuf);
                    friendlyByteBuf.writeVarInt(integer);
                    friendlyByteBuf.adventure$locale = channelHandlerContext.channel().attr(PaperAdventure.LOCALE_ATTRIBUTE).get(); // Paper
                    Bukkit.broadcastMessage("Deep clone packet. -> " + ByteBufUtil.getBytes(org).length + " " + friendlyByteBuf.readableBytes() + " -> " + friendlyByteBuf.capacity());
                    clone.put(ByteBufUtil.getBytes(org));

                    //20.08 17:09:18 [Server] INFO Deep clone packet. -> 45 1 -> 256
                    //20.08 17:09:18 [Server] INFO Deep clone packet. -> 45 1 -> 256
                    //20.08 17:09:18 [Disconnect] User PedroException has disconnected, reason: Internal Exception: io.netty.handler.codec.EncoderException: java.nio.BufferOverflowException
                    return;
                }

The // are logs from the server, the main problem is, why would it print a buffer overflow, if the data was sent in a previous packet

[chrisvest]

You're writing to the buffer, but it's too small for the data.