Enhancement: Variable protocol field positions depending on LENGTH type fields

[jacobagilbert]

A common protocol structure is to have a header containing a length field, followed by data of length bytes, followed by additional protocol fields such as a CRC. If the length field is variable, the position of subsequent URH-specified protocol fields defined by fixed position will not be in the correct location.

I do not know the best way to address this, but quick and easy way to support this would be through allowing field values to be used as variables in the Start and End location columns in the Edit Protocol Labels from Message Type UI Window. This is not a great solution because of the number of edge cases. A better option might be to create an Advanced Settings option where LENGTH type fields could specify which fields they apply to and the behavior of downstream fields. Alternatively, DATA type segments could have an Advanced menu where their length could be assigned to depend on the value of a LENGTH type field.

The CHECKSUM type advanced menu is incredibly useful and expanding this could be very powerful for other field types.

[jacobagilbert]

I've been working with this a little more and I think that almost all use cases could be addressed by allowing LENGTH type fields as variables when specifying field start/end addresses. This even works with protocols with more complicated structures having more than one length field e.g.: TLV header components or multiple data sections.

I am pointing this out because it seems to me like the easiest (and least impactful) way to support this to me, but I need to get more familiar with the code.

[andynoack]

Thanks for your ideas! We discuss them internally but as you already pointed, internal dependencies will probably be a problem (e.g. copy and paste, undo, ...). At the moment we address the problem with different message types as for most IoT that we have seen protocol messages of one message type have a constant length. Therefore you would create different message types and assign them automatically by the value in the length field. Have you tried this feature?
We had also discussed earlier to make labels be align-able relative to the end and not the beginning of the message (CRC) but omitted it because we have decided to go for the message type thing.
Additionaly as we may have a lot of wrong values in the received messages, a dynamic alignment can make things very complicated.
What do you think?

[jacobagilbert]

@andynoack - I have currently been doing exactly what you have suggested, but there are some real limitations doing this when you have many or unpredictable length values. Mostly that it becomes extremely easy for different message types that describe the same actual framing structure to become out of sync from each other, but also that there can become an unmanageable number of such messages quickly. Both of these issues are greatly amplified if you are actively attempting to understand a protocol and want to assign new fields.

I have continued to think about some of the edge cases and internal logic and my current best idea remains the use of relative offsets for protocol elements, with the possible addition of an 'Advanced' menu for LENGTH type fields that allows for bounding the LENGTH type field values, and potentially accepts a function to assign that value to a specific variable that could be used for other fields. Something like this:

Assuming this is a reasonable solution, the main issues I currently see with this are:

• Implementation of using variables in field position table
• Possible of a length field location self-reference (solution: logic to prevent self-reference)
• Behavior of dependent fields when the referenced length field is deleted (solution: just delete these fields too? default variables w/o valid references to 0?, flag fields as in error to users?)
• What to do when length fields are invalid (solution: URH already handles out-of-bound fields gracefully, just accept this?)

Thoughts? You both know the code much much better than I do. If this is something you aren't interested in, that is ok too.

[jopohl]

I think pimping up the Protocol Label Dialog is a nice idea but I would go for a more general approach:

• Make every label referable and not only the length field. So you could, e.g., set the start of the synchronization to preamble.end + 1.
• In the same spirit, refer to the value of fields with syntax like length.value so in your example the end of data would be 19 + length.value.

With this approach, it would not be necessary to add advanced settings for the LENGTH field. Every label would get a start, end and value property.

However, the main issues you are referring to remain mostly valid for this approach as well. Another point to consider is that if the value of a field changes (e.g. length value is edited or refreshed), all depending labels must be updated as well. This will require a significant amount of work in the backend. Basically we would need a kind of proxy which transforms these relative start/end values to absolute ones.

[jacobagilbert]

I think this makes a lot of sense, thank you for the input. I'm going to take a look at this in my spare time; if anything comes to you in the mean time ill be checking here.

[adentse]

I was looking for this feature but found this. I just want to add that I would like this because I'm looking into a protocol wich I'm trying to understand. I know length and i know that checksum is always after the data. Length varies and sure I can set up many message types, but right now I would want to explore a bit and knowing which messages has a valid checksum is a good start i want to ignore the broken ones.

As a side note, maybe it would be nice to be able to hide messages with invalid checksum? (maybe it's already possible?)