Native units support

[KOLANICH]

Often values in fields are in some units. Either dynamically selected ones, or hard-specified. They are definitely a part of documentation and implementations, so they definitely should be added to ksys.

Currently the only way to add them are freeform fields (the ones prefixed with a dash) or docs or comments.

But units and quantities in fact are supported in different programming languages. Some with 3rd-party libs, some units are available in stdlibs. The libs do conversions automatically and protect programmers from unit-related errors.

So it may be useful to have a built-in units support. Units are to be added into unit property. That property should allow both hardcoded and selectable (switch-on - cases) units. By default the property is ignored when generating source code (i.e. C++ or python), but takes effect when generating docs i.e. doxygen or docs in source code. But a user may enable creating unit-typed code with an argument. The argument is the name of one of the supported unit types libs, for example pint for python. Internally in the compiler it maps into an object unitLibs in RuntimeConfig, containing references to UnitLib objects for each language. UnitLib is an object defining remapping between KS unit names into library-specific unit names. It has a wrap(KSSpecificUnit unit, string expressionToBeWrapped, UnitConversionType type) method, wrapping the passed expression into the code creating specific unit object. KSSpecificUnit is the type for KS specific units. SI prefixes should be supported. UnitConversionType is a enum defining the type of conversion. At least 2 values are needed, one for assigning a unit, another one is for conversion into that unit.

[GreyCat]

Thanks, this is a good idea. We have something easy to start with: it might be unit keyword that is only for structured documentation purposes, and further on, we might introduce some more magic with conversion into it, etc.

My only concern is that we should start with a decent table of "recommended" unit designations, so everyone will be using the same (and not px, pixel, pixels, pix, etc). I can add compile-time warning if a unit is used that does not match the recommended list.

@KOLANICH, would you like to champion creation of such a list?

[KOLANICH]

IMHO, before creating a table we should develop guidelines for this table.

IMHO:

• support both succinct and full names, using the official abbreviations present in standards
• succinct names are used with succinct prefixes, full names are used with full prefixes.
• use singular, not plural
• we also will need some mechanism to allow a user to define own derived units as expressions in KS expression syntax and own base units. So we need an abstraction for registring a unit too.

I can add compile-time warning if a unit is used that does not match the recommended list.

IMHO it's better to forbid that at all, because it almost certainly will cause ah error. If a user needs an unsupported unit, he either

• adds support of this unit into compiler units libs.
• defines an own base or derived unit in ksy.

[GreyCat]

Um, s/succunt/succinct/?

• support both succinct and full names, using the official abbreviations present in standards

Agreed.

• succinct names are used with succinct prefixes, full names are used with full prefixes

If you mean something like appending automated prefix/suffix to attribute IDs, this could be the next step.

• use singular, not plural

Seems to make sense.

• we also will need some mechanism to allow a user to define own derived units as expressions in KS expression syntax and own base units. So we need an abstraction for registring a unit too.

Again, this could be steps N+2, N+3, etc. For now, I'd stick with the very basic implementation.

IMHO it's better to forbid that at all, because it almost certainly will cause ah error.

Eventually, yes, but as a start, I'd stick with a very simple solution, basically doc-only, which eventually can grow into code generation, etc.

[KOLANICH]

Um, s/succunt/succinct/?

Yeah, really. Fixed, thanks.

If you mean something like appending automated prefix/suffix to attribute IDs, this could be the next step.

I mean prefixes like kilo and kibi. km or kilometer, not kilom or kmeter. Prefix impl is meant to be automatic - if we have a unit W = Watt, we get YW = Yottawatt and YiW = Yobiwatt automatically.

One more thing. American spelling: meter, not metre.

[GreyCat]

Ok, now it all totally makes sense. I wonder if @ams-tschoening might benefit from this proposal too — AFAIR, he worked with certain values like voltages/currents.

[ams-tschoening]

I'm not sure that I fully understand what the goal of this enhancement actually is, but if it is what I think it is ;-), while it might be of some help for me, my use case seems pretty complex and I have the feeling it's the worst-case scenario for your feature.

I don't necessarily have a 1:1 mapping of some bits meaning voltage or current, but in most cases some bits define a combination of some unit and a multiplier for some range of values within that unit as well. Additionally, the number of bits for some unit might even be variable and to make things worse, units can be combined to create some officially documented unit with some special meaning or even some unofficial one only relevant to some implementer.

So the approach I'm currently implementing is parsing the bytes meant to describe some unit, stripping multiplier/range info and ultimately combining all normalized unit bytes into some large table with units I need to support currently only. That table is then switched on in higher level Java to actually calculate things, this time including multipliers etc. to normalize values.

So, how I understand this enhancement is that you are actually trying to introduce language support for the map I'm creating manually currently? I hope the following screenshots make more clear what I'm talking about.

[GreyCat]

@ams-tschoening Thanks for bringing this up! Actually, yes, it looks very much like that would be a very complex application of this proposition, but eventually we might get there too. One important extra aspect that you've mentioned here is that this "unit" thing absolutely needs to have a multiplier/divisor thing, offset, or may be even some more complex logic (like exponential scale?).

Ideally, probably having something that is guaranteed reversible (so it would work both for reading & writing) should be great. I'll need to read/ask more about that.

[KOLANICH]

in most cases some bits define a combination of some unit and a multiplier for some range of values within that unit as well.

I encountered a much simpler case: 1 byte is for a unit (i.e. nm, μm, cm^-1), then follows an array containing numbers (number type is determined by yet another tag). In ksy I don't do any unit conversions, but I have assigned the enums the names, which can be parsed into units, in my lib (it is written in python, and the library implementing units I use is pint) I reverse enums values into their string names, parse name into a unit object, normalize unit, and multiply (not always, if I need an array, I use it as a raw array of numbers, but if units are used to specify range, I use quantities objects) that unit on number to get quantity.

Scale multipliers and offsets fit here nicely only if they are hardcoded and have meaning. This is because quantities work the way similar to types - one can arythmetics with them, libs and compilers should do necessary checks and conversions themselves (ideally - in compile time, if it is possible). Per-blob multipliers and scales IMHO should be dealt with instances.

Additionally, the number of bits for some unit might even be variable and to make things worse, units can be combined to create some officially documented unit with some special meaning or even some unofficial one only relevant to some implementer.

It is not a problem: the libs do necessary conversions and reductions themselves on arythmetics.