Support of multiple measurement units in VSS signals

[nickrbb]

In PR#669, a discussion was started on the issue of different units of measurement being used by different customers. To recap, current we define a "default" unit, which is mostly, although not always, in metric. However, there is a large proportion of the world that doesn't use metric (or whatever default is specified) and so the default unit will not suit them. A common example is that of vehicle speed, where seemingly one half of the world's countries use kilometres per hour and the other half use miles per hour. This results in unit conversion always having to be done at entities that requested the signal and received a response in a different unit to the one that they prefer/require e.g. kilometres per hour to miles per hour.

In a client-server model, this effectively means all clients having to implement code to do such conversion, whereas it would be more efficient to have the server do the conversion needed and save duplicate code amongst client applications. BlackBerry has also observed that some in the industry do not follow the COVESA specified default unit and instead use their own preferred unit instead. This requires client applications to "know" that the signal information returned in a response contains a unit of measure different to that defined in VSS.

Therefore, it is proposed to enhance VSS to accommodate different preferences (potentially on a per regional basis) of measurement units rather than instil a single unit of measure on the whole industry, as we effectively do today. To accomplish this, it is proposed to enhance signal definitions to include multiple measurement unit types as part of the VSS signal path name. This then enables VSS to accommodate for a plethora of different unit types, and potentially also simplify client pre-configured "knowledge" of systems and unit conversions.

As an example for this, consider the Vehicle.Speed signal. This could be extended to include all possible units specified in the defined speed "quantity" (previously called "domain") e.g. Vehicle.Speed.mph, Vehicle.Speed.kmh. Or to put it another way, we introduce branches to signals, providing one branch per possible measurement unit type as defined in the units.yaml file. This could even be automated via vss-tools e.g. every time vss-tools finds a measurement unit domain indication, it could replace it with branches associated with each measurement unit; Vehicle.Speed.[quantity:speed] -> Vehicle.Speed.kmh, Vehicle.Speed.mph.

This way, a VSS "client" can request the unit it prefers and the VSS "server" can take care of any needed conversion and provide the requested signal in the requested unit. If no unit is provided, then we could also allow the default unit (as we already specify) be used instead. This would also allow for some backwards compatibility.

Comments welcome.

[doganulus]

I think there is confusion regarding the role of the standard catalog here. Despite all the good hopes and intentions, the standard catalog cannot be used universally, as I try to explain here. Different systems will have different signal specifications, but hopefully, all standardized by the VSS.

So system developers will define the speed signal with their desired unit. The clients will understand which is which via vspec or schema files.

[erikbosch]

I see two possible paths here. One is to explicitly specify already in source *.vspec file which units that are supported for a specific signals, the other is to rely on "dynamic unit support" based on for example unit files. If we go for the latter approach nothing needs to be changed in the source *.vspec files themselves, it is up to the protocol or backend to provide a method to request the signal in different units, if the backend or protocol supports this. A typical example could be VISS/HTTPS, where it would be possible to use calls like:

Call	Result
GET /Vehicle/Speed HTTP/1.1	Get speed in default unit (km/h)
GET /Vehicle/Speed/km/h HTTP/1.1	Get speed in km/h (we may need to discuss if we should allow / in unit identifiers, makes life more complex)
GET /Vehicle/Speed/Knots HTTP/1.1	Get speed in knots
GET /Vehicle/Speed/Mph HTTP/1.1	Get speed in miles per hour

Then one could of course discuss how a VISS-server would be able to support this. Is it it sufficient to get the *.vspec files as they look today and then a separate unit/quantity file, or would implementation be simplified if vss-tools could generate an "expanded VSS model" where all possible paths are present?

My view for now is that I have some doubts on if it is a good idea to explicitly specify multiple supported units for a signal. We could easily end up in long discussions on usable units for every individual signal. Like should we support mm and cm for Vehicle.TripMeterReading?, or only m and km. Isn't it better if an implementation either only supports the default unit (today km) or supports all units of quantity length?

[nickrbb]

Thanks @doganulus and @erikbosch for your comments.

@doganulus I agree that system developers can use implementers' vpsec files to tailor their applications for the target system. However, such tailoring takes time, money, and resources, so if we can simplify the needed tailoring, then surely that is a positive outcome for the industry?

@erikbosch The use of vss-tools to generate the expanded model in VSS definitions was certainly my preference. Regarding what multiple units would be needed, I agree and share your concern. We certainly don't want absolutely every possible measurement unit being provided on the off-chance that someone might need it in a road vehicle. To that end, I think we should limit the measurement units to those that have a clear and significant market need. Going back to the Vehicle.Speed example, there is a clear and significant market need for the measurement units of MPH and KM/H, with different regulatory authorities in different regions around the world mandating speed be provided to vehicle drivers in those units. For vehicle speed to be provided in knots, I know of no regulatory authority that requires this for a road vehicle (I am willing to be educated here though! :-) ). Similarly, is there a clear and significant market need for mm and cm to be used in trip meter readings? Again, limiting ourselves to clear and significant market needs would suggest no.

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units. I was not looking to expand VSS to provide every single possible measurement unit, as I agree that that would severely overly complicate implementations e.g. VISS servers. To that end, we could also perhaps agree to limit ourselves to a maximum of say 3 default measurement units per signal. Of course, some signals may still be OK with less e.g. a default of 1, based on market need, and that is fine.

[doganulus]

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units.

Hi @nickrbb, assume there is an integer on the speed signal. How should we interpret it? Unless specified otherwise, this value is interpreted using the default unit. If defaults are multiple, we still do not know how to interpret signals. So, they must be singular values by definition.

If different API endpoints are desired, then they must be defined as separate signals, with each having a default unit. This is fine for custom VSS server implementations, and for the standard catalog if the community agrees. However, it is less elegant and creates a precedent for numerous similar cases, which may be problematic in the long run. I have seen several projects going that way and getting bloated at the end, unfortunately. Therefore, it is better to avoid it, at least for the standard catalog.

A better solution to the issue is to standardize an overriding mechanism in the VSS. That amount of tailoring, I think, is unavoidable, even if I agree that less tailoring is much better.

[nickrbb]

Perhaps I should clarify my above posting here in that I was looking to just expand on the "default unit" paradigm i.e. enable VSS to define multiple DEFAULT measure units.

Hi @nickrbb, assume there is an integer on the speed signal. How should we interpret it? Unless specified otherwise, this value is interpreted using the default unit. If defaults are multiple, we still do not know how to interpret signals. So, they must be singular values by definition.

If there is only one signal for speed (i.e. Vehicle.Speed), then yes, we need to specify a default unit, and hence we have the issue I describe above about what what measurement unit should be picked. Having multiple defaults for one signal should never be done, and just to be clear, I'm not suggesting that.

If different API endpoints are desired, then they must be defined as separate signals, with each having a default unit.

Agreed.

This is fine for custom VSS server implementations, and for the standard catalog if the community agrees. However, it is less elegant and creates a precedent for numerous similar cases, which may be problematic in the long run. I have seen several projects going that way and getting bloated at the end, unfortunately. Therefore, it is better to avoid it, at least for the standard catalog.

Well, that's kind of a philosophical debate, and I'd rather stick to the problem in hand here. Whilst I agree that bloat can be an issue in any project, I have also seen projects fail because they fail to take into account market needs.

A better solution to the issue is to standardize an overriding mechanism in the VSS. That amount of tailoring, I think, is unavoidable, even if I agree that less tailoring is much better.

The overriding mechanism in VSS is what we have today and is what is causing the issue. The proposal here is to more gracefully allow modifications to VSS by extending certain branches and therefore encouraging their use/non-use, rather than encouraging their overriding which is causing the issue I describe above.

Maybe I'll put some slides together for our call today, to clarify.

[erikbosch]

Added it as a topic for the meeting today , see https://wiki.covesa.global/display/WIK4/VSS+Meeting+Topics+and+Meeting+Notes

[erikbosch]

Meeting notes:

Nick presented the topic

• VSS default one unit as default

• We want to provide multiple branches for units that have "significant market needs", like Vehicle.Speed.Mph and Vehicle.Speed.Kmh

• We shall be conservative on specifying units, possibly max 3

• Keep Vehicle.Speed as path, possibly reply as Vehicle.Speed#Mph

• Ulf: VISS provides solution - you can request metadata (signal discovery) and then get what unit is used by server.

• Pierre: By coding it into catalog you could see it in advance

• Adnan: If we handle different units we handle it on API level. But it would be interesting to see automatic translations

• Ted: I see this as a display domain, there are solutions like https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation

• Scott: If a OEM change unit on server side it is their fault that they cause problems

• Erik: How to handle it VSS

• Nick: Possibly one leaf node per unit, based on unit files, possibly flag.

• Erik/Nick: Could be handled by vss-tools

• Ulf: think it should be handled on API level, could be included in API call. Server respond with error if it does not support type.

• Pierre: How would that be done?

• Ulf: For example, could be a hash after signal name in VISS,

• Pierre: So specify unit is call?

• Adnan: If we have domain/quantity info we can transform, no need to change tree

• Daniel: Can create a requirement that pub/sub shall be possible to be performed in any unit

• Erik: if your API supports attaching unit when you send/receive values then you do not need to change paths.

[doganulus]

Thank you for the today's discussion. I think there is a valid use case for @nickrbb's case but I am unsure about expanding the signal tree (leaf nodes) by units, but the other ideas are more interesting.

I think the API fragment (#) solution is nice. The query syntax (?) is standardized in the URIs and we could write it such as /vehicle/speed#?format=mph. Moreover, this may handle accessing struct elements such as /a/vss/struct#/its/nested/property using the JSON pointer syntax. Overall, I think this idea could work and is open to expansion.

Besides the solution above, I think tailoring the catalog for different cases is unavoidable in general. Maybee manufacturers might want to prepare different vehicle.vspec files for Europe, North America, Japan markets etc.. This may be done for each vehicle model as well. But it's eventually up to the client to read vehicle.yaml and configure themselves for different cases automatically.

We should use VSS files for system and app configurations.

[nickrbb]

Support of multiple measurement units in VSS - 2023-11-21.pdf

Please find attached the slides I presented yesterday.

Just to reiterate, the problem is that of identifying the measurement unit of a received signal. It is not the issue of conversion, as some on the call may have misunderstood yesterday.

[doganulus]

the problem is that of identifying the measurement unit of a received signal

Could you please elaborate more on it with different words? In my understanding, we have a pub-sub system, and there is a topic name /vehicle/speed. We want to interpret raw numeric values on the topic and pick a default unit for the topic. Then:

• Definitely, I wouldn't want duplicate topics in my system just because of different units. This creates unnecessary bandwidth and synchronization problems in the network. There should be a single speed signal across the whole system. Then handling different units is a conversion problem.
• Or, you may override the default unit for your own system (with your own vspec files). In this case, clients would read vspec files and learn the default unit for each signal. But you didn't seem to like tailoring either.

What's the third way?

[erikbosch]

Meeting discussion:

Nick

• should this be solved in protocol or in API?

• We have experienced problems where OEMs have changed unit, so data in wrong unit is returned

• So suggesting measurement unit to be part of path

• All up for the different solution, presentation published, please provide feedback.

• Achim: Related to AUTOSAR config

• Adnan: Related to our conversion discussion

• Nick: Bit client needs to be able to provide what unit it expects.

• Adnan: Will create a small presentation for next meeting. Lets discus next week

• Pierre Pierre: The presentation from Nick describes the problem,

Continue discussion next meeting (Dec 12th), to see what solutions that the community seems to prefer

[erikbosch]

Thought a bit more on this issue, and tried to structure my thoughts. Maybe we can use this as a decision structure, as I see different aspects of the discussion

Do we need to change VSS "core" syntax (the syntax used in *.vspec files in this repo)

• I do not think so. With a default unit for each signal we could automatically identify alternative units from the unit/quantity-files.

Do we need to change the syntax or content of unit/quantity-files

• Possibly to specify conversion factors between different units, makes it easier to implement conversion for custom units or units were there is a risk for ambiguity (like horsepower, do we mean mechanical horsepoer, metric horsepower, or electrical horsepower)
• Possibly to specify recommended units. We could for example add recommended: False or auto: False to horsepower as indication to VSS implementations that they do not need to offer signals with quantity "power" in horsepower unless default unit is horsepower.
• Possibly to get unit identifiers that are friendly as identifiers. An identifier like m/s^2 may cause problems. Either we change to something that fits most environments (like only [A-Za-z0-9_]), or we leave to the API to escape or convert the identifier if needed. But that could give problems if we in VSS would have units that only differ by special characters, like m/sand ms.

Do we need to change what VSS says about APIs?

VSS is in itself not an API, but as part of VSS-project we do state some assumptions on APIs and by tooling we in some way set the standard, like:

• We specify how instances shall be expanded as branches and as dot-notated identifiers like Vehicle.Door.Row1.PassengerSide.IsOpen, and we implement this expansion in vss-tools
• We state that actuators shall have a dual nature (current value and target value) and that a set in an API (unless otherwise specified) shall set the target value and that a read (unless otherwise specified) shall give the current value.
• We state (or at least assume) that unless the API provide a unit mechanism values should be given in the default unit specified in VSS.

With this said - there is nothing prevent implementations like Kuksa to offer an API where you could access for example Vehicle.Speed in Miles per hour by using Vehicle.Speed.Mph as identifier. But we could be more specific about this and add an "API recommendation" to the VSS documentation, maybe something like:

If an API use dot-notated VSS names and supports accessing a signal with alternative units, the identifier <VSS signal name>.<unit> (Like Vehicle.Speed.Mph) may be used by APIs to refer to the signal with the specified unit.

Example:

get Vehicle.Speed # Shall give current speed in the default unit used by the server, if using standard VSS km/h
get Vehicle.Speed.mph # Shall give current speed in miles per hour

It is optional for an API to support alternative units, but if it does it must make sure that the values are synced.

Then it is up to whoever wants alternative unit support to influence VISS, Kuksa and other APIs so that they support it.

Do we need to change vss-tooling

• Even if we state that <VSS signal name>.<unit> is an "approved" (but optional) identifier, we do not necessarily need to change output of vss-tools. An implementation like Kuksa could use existing JSON output plus unit/domain file and get sufficient information from that to be able to support Vehicle.Speed.Mph
• But if requested we could provide a "unit-extension" feature where we expand the tree so that every signal gets represented in every unit supported. But if so we need some annotation or other mechanism that says that Vehicle.Speed.Mph actually is the same datapoint as Vehicle.Speed. It could be deduced implicitly. The server that receives a request for Vehicle.Speed.Mph could check if Vehicle.Speed is a branch or a signal. If it is a signal it would know that Vehicle.Speed is the master-signal and this is just an alias in a different unit.

Performance and Implementation aspects

There were concerns that you might get duplicate topics (if using MQTT) when supporting alternative units. That could be the reality if clients are allowed to use any unit they want and they all interact directly with the topic and the MQTT topic API uses different topics for different units. That may be a problem in some implementations, but not necessarily in all.

Even if you use MQTT you could have an API where the dot-notation only is used in the programmatic API/SDK, the SDK may when getting the request below ask server for the actual unit for Vehicle.Speed and do conversion on client side and use Vehicle.Speed when communicating with the server.

subscribe("Vehicle.Speed.Mph",*MyCallback)

So the transport API does not not necessarily need to know or handle the <VSS signal name>.<unit> convention, it could exist on programmatic level only. So how it is handled in implementations could be quite flexible.

[ppb2020]

@erikbosch, I like your approach, so I'll poach it to provide my feedback... ;-)

Do we need to change the syntax or content of unit/quantity-files

I agree we do not need to change the VSS syntax. All of this can be handled in the APIs, in whichever way the API author determines is best for that API.

Do we need to change the syntax or content of unit/quantity-files

On this topic, I must admit that I am challenged to understand the rationale behind what is currently being discussed outside of this issue regarding units, domains, factors, conversions, etc. Although I understand the concepts being shared and associated discussion (I understand domains, factors, etc.), I need to spend some more thinking time grokking the rationale behind the need to support domains, factors, conversions, etc. in the unit file. Thus, I'll hold comment on this part until I am more comfortable understanding rationale.

Do we need to change what VSS says about APIs?

In my mind, VSS should say very little, if any, about APIs. I can see it making statements like: "This approach or restriction was chosen in order to allow for more flexibility in how APIs are designed and implemented.".

The specification should generally avoid making recommendations about APIs, leaving as much latitude as possible to their implementers. Indeed, a separate project, such as Kuksa Val could certainly make certain recommendations on how to represent requests for signal values in a specific unit as well as what unit (scale, datatype, etc.) conversions it supports.

Do we need to change vss-tooling

I don't see the need to.

The "unit-extension" idea you have is interesting, but I do not see it as necessary based on my statement above that I'd leave all of this to the API implementation.

Performance and Implementation aspects

Again, I'd leave those concerns to the API implementation. Presumable, implementers would select the approach that best benefits their needs and achieves their performance goals.

[erikbosch]

Meeting notes:

• Seems to be conclusion that this better shall be handled at API level, for example in VISS
• Stephen: Could be good to document why this is not a topic for VSS but rather for APIs (design guide)