More flexible and robust DSDL verification.

[kjetilkjeka]

Introduction

Warning: This is a version 1 protocol blocker.

This proposal supersedes OpenCyphal/public_regulated_data_types#36

What

This issue proposes changes to what properties should be verified when receiving a protocol 1 frame compared to a procotcol 0 frame.

More specifically, it argues that we can do more statically to gain some flexible at run time.

Background

It's based on:

• The "DSDL serialziation compatibility analysis" proposal (DSDL serialization compatibility analysis #40)

Why

To acheive the flexibility of doing small (bit compatible) changes to dsdl definitions without breaking existing systems. Without adding much extra work or complexity and keeping the same level of robustness.

Current situation

When the current version of uavcan was created there was no versioning of dsdl messages. There still was a need to change DSDL types after trial and error and the data type signature was used to initialize transfer CRC was used to guarantee that compatible definitions was used.

There are two problems with the current approach that this proposal addresses. The first is that only multi frame transfers are considered in this verification. The other is that some versions of the messages are "similar enough" to be exchanged but fail to be received due to a CRC error.

The proposal

Check more things statically

We write a tool to check the following properties (I currently have the time to write it, but then it will be in Rust).

• DSDL definitions must be immutable - If a version 2.0 is published it should not be possible to change without incrementing the version number.
• Checking binary compatibility within the same major version - Fields (other than void fields) cannot be split up, change type or change name withing the same major version.
• We can extend this tool to verify more properies as needed.

Compatibility tag/signature

Let us introduce something called a compatibility tag and a compatibility signature. The compatibility signature is a less strict analog to the data type signature. It is defined in a way that ensures that two definitions only have the same compatibility tag and compatibility signature if, and only if:

• They have the same path (name and namespace).
• They have the same major version (and minor version if major version == 0). (This implicitly requires a lot of properties to be true. And we're going to have a tool to check this)
• They have the same serialization layout. This is already required implicitely from requiring the major version to be identical. It is mentioned again as it will be enforced explicitely by including the serialization layout in the compatibility tag.

The compatibility tag for a dsdl definition is given as:

<namespace>.<name>.(<major version>|0.<minor_version>)
<serialization_layout>

The compatibility signature is the product of the uavcan signature hash function to a compatibility tag.

Using the compatibility signature

The core of this proposal is to use the compatibility signature instead of the data type signature for CRC initialization for the next version of the protocol. This will make compatbile versions of the same messages compatible in uavcan implementations "out of the box".

Alternatives

Attempting to serialize multiple times

We have talked about attempting to deserialize messages multiple times with different signatures to acheive the same. The benefits of the solution proposed here is:

• Gives more robustness for single frame transfers due to static checks.
• Is more predictable for multi frame transfers due to not having to check some amount of data type signatures for each transfer.
• Makes it easier to implement as dsdl compatibility have been a part of the uavcan version 1 from the bottom up instead of added as an after thought.

Only use version number as dynamic compatibility verification

Do not include data type signature or compatibility signature in the CRC, only use the major (and minor if major_version == 0) version number and rely on static checks for the rest. This makes #40 not needed and will spare the user for concepts as serialization layout.

[thirtytwobits]

it will be enforced explicitely [sic].

How do you envision this is enforced?

[kjetilkjeka]

How do you envision this is enforced?

By including the serialization layout in the compatibility tag it is directly enforced that the compatibility tag can only be equal if the serialization layout is equal. Serialization layout is defined in #40

I also updated the comment in an attempt to make it clearer, thanks for pointing it out.

---

For readers that have not followed all our discussions tightly: I think that this proposal (and #40) have slim chances of being accepted. It looks like we're looking for a more explicit way to convey version info (that not only works for multi frame transfers) and at the same time decouple CRC from message compatibility.

Follow the latest updates of OpenCyphal/public_regulated_data_types#35 for more information