Generating constants that expose the wire-size of SBE variable length data fields

[RichardWarburton]

If I want to use SBE in combination with Aeron's BufferClaim API then I need to know the size of the message that I want to claim. If the message that I want to encode has no variable length data fields then this is easy, just claim the BLOCK_LENGTH. If I have variable length data then I need to claim BLOCK_LENGTH + the sum of the lengths of variable length + the sizes of the length fields for each variable length data.

The sizes of the length fields are stored your SBE's xml schema file, but there is no way from code to get to them without hard coding them in your codebase. If the constants were simply generated as part of the encoders I think it would improve usability of SBE when combined with the BufferClaim API.

Again for me this is a nice-to-have rather than a high priority issue, however, if you agree with the concept but don't have time to implement I'm happy to contribute a patch.

[tmontgomery]

The VarDataEncoding and GroupSizeEncoding classes are generated. So, it should be possible to know those aspects and write something that is not specific to the XML.

[RichardWarburton]

Thanks @tmontgomery, I wasn't aware of those. Looking at the VarDataEncodingEncoder/VarDataEncodingDecoder that I have I don't see any fields that correspond to the size of the length field. There's an ENCODED_LENGTH constant, but its -1 for me.

[mjpt777]

The decoder for var data has the length accessor generated but this is not in the encoder. We could add this.

[RichardWarburton]

Just to clarify I'm not talking about just having access to the encoded length of the field. I'm talking about the encoded size of the length field. For example in the following case its 2:

        <composite name="varDataEncoding">
            <type name="length" primitiveType="uint16"/>
            <type name="varData" primitiveType="uint8" length="0"/>
        </composite>

[mjpt777]

Yes that is what I assumedyou were asking. Look at the CarEncoder and CarDecoder in the samples. The Decoder has it but not the Encoder.

[RichardWarburton]

Perfect, thanks.