Why flatbuffers instead of capnp?

[jbenet]

You mention memory efficiency as the main selling point of flatbuffers. However, capnp already exists, with zero copy "serialization," and a range of compelling features (RPC, etc). Still you saw the need to build flatbuffers, which means this library either (a) does something capnp doesn't, or (b) beats capnp in some respect. Could you please describe somewhere what this rationale is? Why should someone pick flatbuffers instead of capnp?

[julienschmidt]

See the Benchmarks section in the docs:

Cap'n'Proto promises to reduce Protocol Buffers much like FlatBuffers does, though with a more complicated binary encoding and less flexibility (no optional fields to allow deprecating fields or serializing with missing fields for which defaults exist). It currently also isn't fully cross-platform portable (lack of VS support).

Besides the comment above, the two are not that similar. FlatBuffers combines the flexibility of Protobufs with the efficiency of Cap'n Proto, in a sense a best of both worlds approach. Which is better depends on your use case.

[jbenet]

Useful for comparisons https://kentonv.github.io/capnproto/news/2014-06-17-capnproto-flatbuffers-sbe.html

[lilith]

@gwvo Could you explain how flatbuffers is more flexible? I'm having a hard time understanding (at an API level) how flatbuffers improves on cap'n'proto. For my use case, perf differences between the two are negligible. Perhaps an example of a common scenario which flatbuffers handles better than Cap'n'proto could shed clearer light on this?

FlatBuffers, like Protobuf, has the ability to leave out arbitrary fields from a table (better yet, it will automatically leave them out if they happen to have the default value). Experience with ProtoBuf has shown me that as data evolves, cases where lots of field get added/removed, or where a lot of fields are essentially empty are extremely common, making this a vital feature for efficiency, and for allowing people to add fields without instantly bloating things.

Cap'n'Proto doesn't have this functionality, and instead just writes zeroes for these fields, taking up space. They recommend using an additional zero-compression on top of the resulting buffer, but this kills the ability to read data without first unpacking it, which is a big selling point of both FlatBuffers and Cap'n'Proto. So Cap'n'Proto is either not truly "zero-parse", or it is rather inefficient/inflexible, take your pick.

[lilith]

Thanks for the clarification! I'd read this, but wondered if there were other aspects to consider. My use case is for representation of a graph of small nodes, so there aren't any god objects likely to grow extensively (and since I'm using it as IPC and not over the wire, space is not a concern).

If you're use case is small, simple, non-evolving objects, you're likely not going to notice much difference between the two. In that case you're better of picking which you prefer based on API, features, platform/compiler/language support, or whatever is important to you.