FlatBuffer zero copying not working as expected. [JAVA, Flatbuffers 1.11.0, OSX]

[anshkhannasbu]

Use case:

I am trying to achieve zero intermediate copying semantics by utilizing flatbuffers in GRPC streaming.

To benchmark correctly, I developed a small client-server program that repeatedly transfers 1MB data (over 100000 times) utilizing flatbuffers over GRPC.

Issue:

What I observed through CPU profiling the client is that a significant percentage of time is spent when creating the message in the application layer; Specifically in the createString() because that creates a copy of the data.

The expectation was that no intermediate copying of data will happen. Am I taking the wrong approach to create a message? Is there a way to use the FlatBufferBuilder’s internal ByteBuffer to write application data directly, or have the internal ByteBuffer refer to an application buffer slice?

Code:
Client

Flatbuffers schema

[aardappel]

The promised "zero copy" with FlatBuffers & gRPC refers to C++, not Java (see e.g. https://grpc.io/blog/grpc-flatbuffers/).

Not sure how you would achieve zero copy in Java. Also not sure what createByteString is, as it's not present in the FlatBuffers code.. is this something in gRPC? Certainly any use of Java strings would make things not zero copy, since they are represented differently from the UTF-8 that FlatBuffers and most other systems use.

@paulovap

[anshkhannasbu]

Apologies for the typo, i meant createString. I also observed a similar slowdown when using createByteVector in version 1.12.0.

[paulovap]

When you are serializing, you are basically copying all the other objects you want to serialize into Flatbuffer internal's ByteBuffer. So, in other words, everything you add to FlatbufferBuilder is copied to the internal buffer. But, it does not generate intermediate objects and only allocates if the internal ByteBuffer is full and needs to be resized.

AFAIK that happens on all managed languages, not only java.

Having said that, in theory, it should be possible to mitigate the copying. FlatbufferBuilder could support some sort of internal linked-list like structure where you could simply append additional ByteBuffers when explicit requested by the user, instead of the current resizing and copying strategy. That could make things a little more complicated but should be doable.

One reference implementation is netty's CompositeBuffer.

That is also why I added a more flexible buffer interface on FlexBuffers, so we can experiment with this optimizations in the future.

Tl;Dr: It does copy. Does not create intermediate objects. Might allocate if the ByteBuffer is full.

[github-actions]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.