Questions about efficient fpp struct definitions and little endian serialization/deserialization

[ReggieMarr]

Hi All,

I have been implementing Endurosat's User protocol for a flight software project and I thought I'd leverage fpp here. I'm mostly looking for tips of how to best architect my data structures.

For example I have the following fpp code:

  constant ESUP_HEADER_ID = 0x50555345

# NOTE that in byte form it will be represented with LE
  struct EsupPacketHeader {
    HeaderId: U32
    ModuleId: U16
    DataLength: U16
    CmdStatus: U16
  } default {HeaderId = ESUP_HEADER_ID}



    struct EsupConfSetSymbol {
        Header: EsupPacketHeader
        CmdId: U16
        TypeId: U16
        Data: U8 @< 1,2,3,4,5 Msp/s (1 byte unsigned char)
    } default {CmdId = CmdId.ConfSet, TypeId = EsupCmdType.SymRate}

    struct EsupConfGetSymbol {
        Header: EsupPacketHeader
        CmdId: U16
        TypeId: U16
        Data: U8 @< 1,2,3,4,5 Msp/s (1 byte unsigned char)
    } default {CmdId = CmdId.ConfGet, TypeId = EsupCmdType.SymRate}

    struct EsupConfSetTxPwr {
        Header: EsupPacketHeader
        CmdId: U16
        TypeId: U16
        Data: U8 @< 27-33 dBm, step 1 db (1 byte unsigned char)
    } default {CmdId = CmdId.ConfSet, TypeId = EsupCmdType.TxPwr}

    struct EsupConfGetTxPwr {
        Header: EsupPacketHeader
        CmdId: U16
        TypeId: U16
        Data: U8 @< 27-33 dBm, step 1 db (1 byte unsigned char)
    } default {CmdId = CmdId.ConfGet, TypeId = EsupCmdType.TxPwr}
    struct EsupCmdCfg {
        Header: EsupPacketHeader
        CmdId: U32}

Note that for each packet we'll have a header which is going to have a the same default headerId however each command has a unique cmdId and typeId.

I've leveraged defaults to set these however it feels a bit awkward since the CmdId and TypeId should really be part of the header. The reason they aren't is that it doesn't seem that we can express nested defaults.

It would also make sense to be able to set constants within a data structure but that also doesn't seem to be allowed.

Is there some way I can improve this ?

Also since the device I'm communicating with uses little endian I end up having to manually shift things like so:

      std::array<U8, esupMaxDataLen> txParamBuff;

      FlightComputer::EsupCmdCfg cmdCfg;
      FlightComputer::EsupPacketHeader header;
      header.setModuleId(MODULE_ID);
      cmdCfg.setHeader(header);
      cmdCfg.setCmdId(FlightComputer::CmdId::SysModeTx);

      constexpr size_t cmdCfgSize = cmdCfg.SERIALIZED_SIZE + 4;
      size_t cmdCfgPaddedSize = cmdCfgSize + differenceToNearestMultipleOf16(cmdCfgSize);
      FW_CHECK(cmdCfgPaddedSize <= txParamBuff.size(), "Need bigger buff", { this->cmdResponse_out(opCode, cmdSeq, Fw::CmdResponse::EXECUTION_ERROR); return; });

      // Make sure the txParamBuff is 0 to start with
      std::memset(txParamBuff.data(), 0, cmdCfgPaddedSize);
      Fw::ExternalSerializeBuffer cmdCfgBuff(txParamBuff.data(), txParamBuff.size());
      Fw::SerializeStatus status = cmdCfg.serialize(cmdCfgBuff);
      FW_CHECK(status == Fw::FW_SERIALIZE_OK, "Error Serializing buff", { this->cmdResponse_out(opCode, cmdSeq, Fw::CmdResponse::EXECUTION_ERROR); return; });
      uint32_t crc = CRC32::calculate(txParamBuff.data(), cmdCfgSize);
      std::memcpy(txParamBuff.data() + cmdCfgPaddedSize - sizeof(crc), &crc, sizeof(crc));
      // Header
      std::reverse(txParamBuff.begin(), txParamBuff.begin() + 4);

      // ModuleId
      std::reverse(txParamBuff.begin() + sizeof(U32),
                   txParamBuff.begin() + sizeof(U32) + sizeof(U16));
      // DataLength
      std::reverse(txParamBuff.begin() + sizeof(U32) + sizeof(U16),
                   txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16));
      // CmdStatus
      std::reverse(txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16),
                   txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16) + sizeof(U16));

      // CmdId
      std::reverse(txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16) + sizeof(U16),
                   txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16) + sizeof(U16) + sizeof(U16));
      // TypeId
      std::reverse(txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16) + sizeof(U16) + sizeof(U16),
                   txParamBuff.begin() + sizeof(U32) + sizeof(U16) + sizeof(U16) + sizeof(U16) + sizeof(U16) + sizeof(U16));

I suppose I could just override the serialization function for the underlying type but I wanted to see if there was a better way (and if not am I able to get information about the order of members from the object)

Thanks so much for the help btw ! I hope that I can at some point formally release this since on the community page/somewhere I think it'd be useful for others.

[SterlingPeet]

It honestly looks like you are trying to re-implement serialization instead of building your own framer/deframer (which, in effect, is a serializer and deserializer) to talk to an external entity. Have you looked at the Comms Adapter Interface documentation yet?

[ReggieMarr]

Hmm so I guess the route would more be to implement some compatible classes in c++ rather than leverage fpp here then ? I had not sent hat before though and that looks like a very useful reference.

I had though support for little endianess would be baked in behind a flag or something but I guess not ?

[SterlingPeet]

Well, the F Prime serialization was written with ensuring autocoded repeatability in mind. The general mindset that I have seen is to do the conversion at the edge of the deployment for whatever interoperable data format you need. That mindset seems to preceed the formal framer/deframer model, which made implementation much easier.

In other words, the internal serializer is meant to prevent you from having to worry about passing serialized data around inside your project. It doesn't save you from formatting your data to match your external interfaces. That fact that it works with the toy GDS software is a clever leveraging of serialization for internal purposes (and very convenient for testing), but rarely-to-never used in real projects.

Taking a peek at the serializable types source, theres a lot of // MSB first comments in there, which supports the idea that it is done one way for internal consistency. So perhaps the answer to your second question is: No, it supports serializing BE/LE hardware into network byte order, automatically.

I suspect what you really want to do is implement a framer/deframer to serialize your data in a way that you can test works with your [arguably kinda weird] LE format. That would match what I have done in similar situations.

[ReggieMarr]

Hmm that strikes me as odd as I know cfs supports little endian interfacing, but in any case the link you sent seems to point at some good mechanisms for implementing this functionality.

Are you able to speak to my question about expressing nested defaults with fpp (or if there's a better way to express that generally)

[SterlingPeet]

As it turns out, my fluency in FPP details is annoyingly low right now. I have been able to offload the FPP writing part to some competent students on my recent projects, while focusing on high-level architecture and low level problem hunting. If he can find a few minutes to comment, @bocchino would be able to definitively answer the FPP question.

As an effect of writing a comparison paper between F Prime and cFS, I actually was looking at some of those BE/LE switches in cFS today. It turns out that these two frameworks are written with different philosophies, and sit in slightly different parts of the FSW trade space. As such, I actually do not find it surprising that there is a difference here. That said, if F Prime supports EDS as a first class citizen, then the requisite Endianness switches will have to be implemented to match. I would bet that for F Prime, EDS support looks a lot like a fancy framer/deframer implementation that reads the EDS files for conversion definitions. F Prime does not support CCSDS protocols internally, although an interesting 3rd party Space Packet implementation was presented FSW Workshop 2024; about 2 weeks ago.