Reconciling the MessageType 0x10 bit between ExtendedLength and scheduled-command cancellation

[glopesdev]

Two proposals in the 2.0.0 milestone allocate their flag to the same bit in the MessageType field of the 8-bit binary protocol. The ExtendedLength proposal at #218 allocates its flag to bit 4, mask 0x10. The scheduled-execution proposal at #15 allocates its Cancellation flag to the same bit. Since 2.0.0 is aiming to be a clean break, we need to settle the bit allocation now rather than discover the collision during firmware implementation. This post records the finding, including a survey of deployed core firmware, both stable releases and prototypes, and a proposed resolution.

The finding

The MessageType field currently uses bits 0 and 1 for Type and bit 3 for Error. The conflicting flags are placed at bit 4. These two flags are orthogonal and can occur together, for example canceling a scheduled extended-length write, so they cannot share a bit. One of them has to move.

What has been deployed in core firmware

We checked whether any released firmware or client already uses 0x10, because that would constrain the decision. The good news is that there is no clash yet in either firmware or clients over this bit.

• The C# client change that adds the flag, Add support for setting message flags bonsai-rx/harp#85, is still open and unmerged. It assigns the Cancellation flag to 0x10, but it is not in any release. The latest published package is Bonsai.Harp 3.6.1.
• The ATxmega delayed-commands work, Added delayed commands. core.atxmega#5, is merged and included in firmware releases 1.14 and 1.15. It encodes scheduled writes and reads as plain timestamped messages with no new MessageType bit, and it does not implement cancellation. So no fielded ATxmega device uses 0x10.
• The core.pico ExtendedLength reference implementation, Extended-length register write support  AllenNeuralDynamics/harp.core.pico#7, uses bit 0x10 for ExtendedLength.

The conclusion is that the conflict is entirely at the proposal stage. We can reconcile it with zero backward-compatibility impact on any device already in the field.

Proposed resolution

We propose two changes:

1. ExtendedLength keeps bit 4, mask 0x10. It already has written specification text in Add ExtendedLength flag to the binary protocol #218 and a working reference implementation in Extended-length register write support  AllenNeuralDynamics/harp.core.pico#7.
2. The scheduled-command Cancellation flag moves to bit 5, mask 0x20. The only artifact that assigns it to 0x10 is the unmerged client change above, so this needs a single edit before that change is released and breaks nothing.

Current bit-allocation

This is the current status of bit-allocation on the MessageType byte after the changes proposed in 2.0.0. See the MessageType field in the current 8-bit binary protocol for the present definition. If this additional information would be useful to support discussion, we could consider extending the current table layout.

Bit	Mask	Field	Status
0-1	0x03	Type	In use. Read is 1, Write is 2, Event is 3
2	0x04	reserved	Available
3	0x08	Error	In use
4	0x10	ExtendedLength	New in 2.0.0, per #218
5	0x20	Cancellation	Reserved for scheduled-command cancellation, per #15
6	0x40	reserved	Available
7	0x80	reserved	Reserved for Flag32 in the future 32-bit protocol

Bits 2 and 6 stay available for future flags, and bit 7 stays reserved so the 8-bit and 32-bit protocols do not collide.

What this means for each core

For the core.pico stack, nothing changes. The ExtendedLength reference implementation keeps 0x10 exactly as written. No rework is needed.

For the core.atxmega stack, no fielded firmware is affected, because the released delayed-commands feature never used a MessageType flag bit. The one point to be aware of is forward-looking. When scheduled-command cancellation is implemented on this stack in the future, it must use 0x20, not 0x10. We will update the specification to reserve 0x20 for cancellation.

Related semantic interaction to track separately

This is not part of the bit allocation, but it comes up in the same comparison and should be written down somewhere. ATxmega firmware 1.14 and 1.15 already interpret any timestamped controller-to-device message as a scheduled command. ExtendedLength writes also carry timestamps. The specification therefore needs to state whether a timestamped ExtendedLength write is scheduled for its timestamp or executed immediately. We suggest resolving this in the device-operation conventions at #221 rather than here.

Related issues

• Add ExtendedLength flag to the binary protocol #218
• Message protocol enhancements for scheduled execution #15
• Device operation conventions for extended-length registers #221

[glopesdev]

@bruno-f-cruz @filcarv @Poofjunior tagging for awareness. Feel free to leave any comments, otherwise I will reconcile as stated in the upcoming PRs which will finalize the protocol v2.

[bruno-f-cruz]

Sounds good to me provided we reserve the bit for future use in the device specification.