Skip to content
This repository has been archived by the owner on Apr 7, 2022. It is now read-only.

WIP for the Properties RFC-5 #30

Closed
wants to merge 9 commits into from
Closed

WIP for the Properties RFC-5 #30

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
52e5214
WIP
gwen-soleil Jul 10, 2019
7902371
WIP on properties
gwen-soleil Jul 10, 2019
b2b0422
WIP (reformulations, typos, e-mail format)
bourtemb Jul 10, 2019
2d6a471
Added Andy Gotz as contributor in properties rfc 5 (#1)
andygotz Jul 16, 2019
ec3172e
Update introduction of 5/README.md
gwen-soleil Jul 18, 2019
9b00d8d
integrate comments from PR #30 + WIP
gwen-soleil Aug 22, 2019
e01dcb8
WIP
gwen-soleil Aug 22, 2019
ab4ed16
Class specification is part of the device RFC
bourtemb Sep 13, 2019
da3d37f
Take into account Vincent Hardion comments from https://github.com/bo…
bourtemb Sep 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
158 changes: 141 additions & 17 deletions 5/README.md
View file
Open in desktop
@@ -1,18 +1,22 @@
---
domain: rfc.tango-controls.org
shortname: 5/PROPERTY
name:Property
name: Property
status: raw
editor: Vincent Hardion (vincent.hardion@maxiv.lu.se)
editor: Gwenaëlle Abeillé `<at synchrotron-soleil.fr - gwenaelle.abeille>`
contributors:
- Vincent Hardion `<at maxiv.lu.se - vincent.hardion>`
- Reynald Bourtembourg `<at esrf.fr - bourtemb>`
- Andy Gotz <andy.gotz@esrf.fr>
---

This document describes the Property, a Tango concept representing an element of configuration in order to customise a Tango element. This document describes version 1.0 of the Property.
This document describes the Property, a Tango concept representing one or more values to configure one of the following Tango elements namely Tango Device, Class, Attribute, System or a not associated to any Tango element, so called free properties. This document describes version 1.0 of the Property.

See also: Y/OtherTemplate
See also: 2/Device, 6/Database, 4/Attribute

## Preamble

Copyright (c) 2019 MAX IV Laboratory.
Copyright (c) 2019 Tango Controls

This Specification is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, see <http://www.gnu.org/licenses>.

Expand All @@ -22,7 +26,12 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S

## Tango Property Specification

A Property is designed to represent any information of configuration in Tango.
A Property is a persisted item of Tango that is mainly used for configuration purposes. 4 types of properties exist:

* Class Property: a Property related to a Device Class (cf RFC-2).
* Device Property: a Property related to a Device (RFC-2).
* Attribute Property: a Property related to an Attribute (RFC-4).
* Free Property: a Property defined for the entire Control System (RFC-1) i.e not related to any Class, Device or Attribute.

### Goals

Expand All @@ -32,7 +41,7 @@ Additionally, it aims to:

* Provide the initial information when one starts a Tango Device

* Be usable as keeping global variable in the Tango control system
* Be usable as keeping global variable in the Tango Control System

* Provide meta data complementing any data in Tango

Expand All @@ -45,23 +54,138 @@ There are many use cases for the usage of a Property:

* To change the representation of an information in order to be more user friendly a Property can define a new data format for the graphical user interface to convert the raw data.

* To configure some alarms on Attribute value.

## Specification

A Tango Property is a strict definition of a pair of key/value
* configuration are represented in the form of properties.
* data are represented in the form of attributes.
* actions are represented in the form of commands.

Its model can be represented as a defined tree which each elements are from a defined types: Class, Device, Property, Attribute, Command following the rules below:
* To memorize the Attribute value each time it changes.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this means "marking an attribute as memorized" ? I'd rewrite this as "To mark an attribute as memorized, and store the value each time it changes".

Copy link
Member Author

@bourtemb bourtemb Oct 16, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lorenzopivetta , I agree it is currently a bit ambiguous. I would not use "To mark an attribute as memorized" because this part is done in the code, not in the DB.
I would reformulate to something like:

  • to memorize the Attribute value of a Memorized Attribute when it changes

The idea here was to talk about the role of the __value attribute property.


## Specification

A Tango Property is a strict definition of a pair of key/value
* The Property SHALL have one key, called Property Name
* The Property SHALL have one value
* The Property SHALL have one value, which could be empty, called Property Value
* If the Device is started upon a Tango Database, Class, Device, Attribute and Free Properties MAY be persisted into the Tango Database (RFC6)
* If the Device is started without a Tango Database, the device and class properties MAY be persisted in a file (but no support for the attribute properties).
* All Device and Class Properties MAY be directly modified and accessed through the database device or its file (TODO: to be moved to Request/Reply RFC)
* Attribute Properties SHOULD be modified and accessed at any time through the Device object (RFC-2) (TODO: to be moved to Request/Reply RFC)
* A Property Value MAY have a default value
* A Mandatory Property is a Property for which the Property Value SHALL NOT be null and SHALL not have a default value
* If the Device is started upon a Tango Database, the Class and Device Properties changes are historized in the Tango Database (RFC-6) (TODO: To be moved to Tango Database RFC)
* When a error occurs (i.e. network issue, Tango Database timeout...), a DevFailed exception MUST be thrown (RFC-x for DevFailed/Common types?) (TODO: To be moved to Request-Reply RFC)
* All Device and Class Properties MAY be loaded at init command or device start-up (RFC-8)
* A Device or Class Property MAY have a default value.
* The Device Property Value loading flow in pseudo-code is (TODO to be moved to RFC-8):
```
property-value = getDevicePropertyFromTangoDBorFile(device-name, property-name)
if(property-value is empty)
property-value = getClassPropertyFromTangoDBorFile(class-name, property-name)
if(property-value is empty && default-value is not empty)
property-value = default-value
```
* The Class property value loading flow is:
```
property-value = getClassPropertyFromTangoDBorFile(class-name, property-name)
if(property-value is empty && default-value is not empty)
property-value = default-value
```
* The precedence of Property Value for a given Property Name SHALL follow this order of preference:
1) the last Device Property Value
2) the last Class Property Value
3) the default Device Property Value
4) the default Class Property Value
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not completely true...
Here are the results of some tests I did:

device prop. value in DB class prop. value in DB device prop. default value device prop. mandatory in DB class prop. default value Prop value at the end of init_device
not defined not defined not defined TRUE not defined Error
not defined not defined not defined TRUE Class Property Default Value Class Property Default Value
Device Property Value in DB not defined not defined TRUE Class Property Default Value Device Property Value in DB
Device Property Value in DB not defined not defined TRUE not defined Device Property Value in DB
not defined not defined not defined FALSE not defined empty string. 0.0 for double (not initialized in the code so, depends on compiler?)
not defined not defined not defined FALSE Class Property Default Value Class Property Default Value
Device Property Value in DB not defined not defined FALSE Class Property Default Value Device Property Value in DB
Device Property Value in DB not defined not defined FALSE not defined Device Property Value in DB
not defined not defined Device Property Default Value FALSE not defined Device Property Default Value
not defined not defined Device Property Default Value FALSE Class Property Default Value Class Property Default Value
Device Property Value in DB not defined Device Property Default Value FALSE Class Property Default Value Device Property Value in DB
Device Property Value in DB not defined Device Property Default Value FALSE not defined Device Property Value in DB
not defined Class Property Value in DB not defined TRUE not defined Class Property Value in DB
not defined Class Property Value in DB not defined TRUE Class Property Default Value Class Property Value in DB
Device Property Value in DB Class Property Value in DB not defined TRUE Class Property Default Value Device Property Value in DB
Device Property Value in DB Class Property Value in DB not defined TRUE not defined Device Property Value in DB
not defined Class Property Value in DB not defined FALSE not defined Class Property Value in DB
not defined Class Property Value in DB not defined FALSE Class Property Default Value Class Property Value in DB
Device Property Value in DB Class Property Value in DB not defined FALSE Class Property Default Value Device Property Value in DB
Device Property Value in DB Class Property Value in DB not defined FALSE not defined Device Property Value in DB
not defined Class Property Value in DB Device Property Default Value FALSE not defined Class Property Value in DB
not defined Class Property Value in DB Device Property Default Value FALSE Class Property Default Value Class Property Value in DB
Device Property Value in DB Class Property Value in DB Device Property Default Value FALSE Class Property Default Value Device Property Value in DB
Device Property Value in DB Class Property Value in DB Device Property Default Value FALSE not defined Device Property Value in DB

This is valid only for C++.
I found some of the results quite surprising.
For instance, if you have a class property with a default value and a mandatory device property with the same name with no value at all in the DB, the device server will use the class default value in this case.
So, the property is defined as mandatory in DB but there is nothing in the DB.

When you have a class property with a default value and a device property with a default value, and nothing defined in the DB for these property, then the device server will use the class property default value.
With POGO, you cannot generate code

Copy link
Member Author

@bourtemb bourtemb Oct 25, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pogo gives an error when you try to generate code for a Java device server with a class property and a device property having the same property name.
For PythonHL, Pogo can generate the code in this use case, but the behaviour is different than in C++ since it is assigning twice a (different) value to the same class member. The first time for the class property and the second time for the device property. So at the end, it is always a device property.

    # ----------------
    # Class Properties
    # ----------------

    prop_with_default_class_and_prop_val = class_property(
        dtype='DevString', default_value="Default Class Property Value"
    )

    # -----------------
    # Device Properties
    # -----------------

    prop_with_default_class_and_prop_val = device_property(
        dtype='DevString', default_value="Default Device Property Value"
    )


Meaning that:
* The last Property Value has precedence over the default Property Value
* The Device Property Value has precedence over the Class Property
* All Device and Class Properties MAY be modified and accessed through the database device when using a Tango Database (RFC-6)
* A Tango Device MUST manage the following default Device Properties, which are called System Properties, see RFC-cache:

| Name | Default value | Description |
| --- | --- | --- |
| poll_ring_depth | 10 | Polling buffer depth |
| cmd_poll_ring_depth | "" | List of command polling history depth |
| attr_poll_ring_depth | "" | List of attribute polling history depth |
| poll_old_factor | 4 | Data too old factor |
| min_poll_period | "" | Minimum polling period for attributes and commands |
| cmd_min_poll_period | "" | List of command minimum polling periods |
| attr_min_poll_period | "" | List of attribute minimum polling periods |
| polled_attr | "" | List of polled attribute's names |

* A Tango Device MUST manage the following default Device Properties, which are called Logging Properties, see RFC-logging:

| Name | Default value | Description |
| --- | --- | --- |
| logging_level | WARN | initial Device logging level (FATAL, ERROR, WARN, INFO, DEBUG, OFF) |
| logging_path | "/tmp/tango-<logging name>" or "C:/tango-<logging name> (Windows)" | Logging file path |
| logging_rft | 20480 (20 Mega bytes) | Logging rolling file threshold in bytes |
| logging_target | "" | Initial Device logging target, a file or a device. ex: "file:/tmp/file.log", "device:log/device/logger.1" |

* An attribute MUST manage the following default attribute properties (TODO: move this to specifics RFCs: Attribute Configuration properties / Attribute Event properties / Attribute Properties?):
* Attribute Configuration properties :

| Name | Default value | Description |
| --- | --- | --- |
| label | "" | |
| description | "No description" | A description of the attribute |
| unit | "No unit" | |
| standard_unit | "No standard unit" | |
| display_unit | "No display unit" | Unit to display |
| format | if attribute type is a string, "%s". If attribute type is float or double, "%6.2f". "Not specified" otherwise | Display format |

* Attribute Configuration properties, see RFC-4:

| Name | Default value | Description |
| --- | --- | --- |
| min_value | "Not specified" | Only for numeric attributes. Minimum allowed attribute value.|
| max_value | "Not specified" | Only for numeric attributes. Maximum allowed attribute value. |
| min_alarm | "Not specified" | Only for numeric attributes. Attribute's value minimum alarm |
| max_alarm | "Not specified" | Only for numeric attributes. Attribute's value maximum alarm |
| min_warning | "Not specified" | Only for numeric attributes. Attribute's value minimum warning|
| max_warning | "Not specified" | Only for numeric attributes. Attribute's value maximum warning|
| delta_t | "Not specified" | Only for numeric attributes. Attribute's value delta time used to process delta Alarm. |
| delta_val | "Not specified" | Only for numeric attributes. Attribute's value delta value used to process delta Alarm. |
| enum_labels | "Not specified" | Only used for DevEnum attributes. List of enumerated labels. |
| __root_att | "Not specified" | Only used for forwarded attributes. Name of the underlying attribute. |
| __value | "" | Only used for memorized attributes. Memorized attribute's value. |

* Attribute Event properties, see RFC-12 and RFC-13 :
| Name | Default value | Description |
| --- | --- | --- |
| event_period | "Not specified" | Change event period. |
| abs_change | "Not specified" | Value to trigger an absolute change event. |
| rel_change | "Not specified" | Value to trigger a relative change event. |
| archive_period | "Not specified" | Archiving event period.|
| archive_abs_change | "Not specified" | Value to trigger an absolute archiving event. |
| archive_rel_change | "Not specified" | Value to trigger an relative archiving event. |

### Naming convention
* The Property's Name SHALL use the following convention:
``` ABNF
property-name = 1*VCHAR
alphanum = ALPHA / DIGIT
underscore = %x5F
device_property_name = 1*1ALPHA 0*254(alphanum / underscore)
class_property_name = 1*1ALPHA 0*254(alphanum / underscore)
free_property_name = 1*1ALPHA 0*254(alphanum / underscore)
attribute_property_name = 1*1(ALPHA / underscore) 0*254(alphanum / underscore)
```
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing with Jive I've found that I can add a property with an empty name as well. And I can also just use a space ( ) as name. There are even no checks for duplicated names so two properties with empty names are possible as well. So I think this is more like property-name = 0*CHAR.

Looking into the database schema, https://github.com/tango-controls/TangoDatabase/blob/master/create_db_tables.sql.in, the name is limited to 255 characters so we can write it as.

property-name = 0*255CHAR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing with Jive I've found that I can add a property with an empty name as well. And I can also just use a space ( ) as name. There are even no checks for duplicated names so two properties with empty names are possible as well. So I think this is more like property-name = 0*CHAR.

This would not be true if we consider properties defined from a file (without Tango Database).
POGO does some checks on the property name and won't accept an empty property name and would remove any space. POGO also forbids device property names with some special characters like '%' for instance which could be interpreted as a wildcard by a Database engine.

Looking into the database schema, https://github.com/tango-controls/TangoDatabase/blob/master/create_db_tables.sql.in, the name is limited to 255 characters so we can write it as.

Indeed, there is this limitation by default.
This is also defined in cppTango for instance:
https://github.com/tango-controls/cppTango/blob/tango-9-lts/cppapi/server/tango_const.h.in#L221

property-name = 0*255CHAR

As we said this morning during the teleconf, this might be true with the current jive implementation (not true when using POGO or a file DB). We will make a note that the current implementation allows this kind of property name (empty name of with spaces and tabs) but this is due to issues with the current implementation. empty property names, property names with tabs, spaces and some special characters (we need to find the list) should be forbidden.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the method which does the check in POGO:
https://github.com/tango-controls/pogo/blob/Pogo-Release-9.6.23/org.tango.pogo.pogo_gui/src/org/tango/pogo/pogo_gui/tools/Utils.java#L404

As I understand this code, only alphanumeric characters are allowed [a-z][A-Z][0-9] and the underscore.
The property shall start with a letter.
An attribute property (and only an attribute property) can start with an underscore.

@srgblnch , @hardion , this could be useful for #7: this POGO piece of code (combined with a slightly deeper analysis) also highlights that "State" and "Status" command names are reserved. Command names seem to be case sensitive because command names "state" and "status" seem to be accepted by POGO.
I've done a quick test generating a C++ device server with a command named "state" returning a DevState and it seems that this use case is not well supported. Pogo generated the source code, but created a method
Tango::DevState MyClassName::state()

AtkPanel did not recognize this "state" command. tg_devtest (jive test device) listed "state" and "State" as 2 commands available on the device but executing the "state" command from tg_devtest did not execute Tango::DevState MyClassName::state() method, but MyClass::dev_state() was executed instead, as for the "State" command.

Similar behaviour when adding a command named "status".

So it would probably be good to specify that "state" and "status" should be reserved|forbidden too as command names.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is indeed a deficiency of Pogo and should be avoided. To be added to RFC on Commands, Attributes and State.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the specifications were not there at the time of implementation, I think it is normal that we fall into those kind of inconsistencies. So I think we should compromise on a specification even though the current implementation does not reflect 100% the specification. I propose:
property-name = 1*255ALPHA

Copy link
Collaborator

@hardion hardion Jul 23, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srgblnch , @hardion , this could be useful for #7: this POGO piece of code (combined with a slightly deeper analysis) also highlights that "State" and "Status" command names are reserved. Command names seem to be case sensitive because command names "state" and "status" seem to be accepted by POGO.
I've done a quick test generating a C++ device server with a command named "state" returning a DevState and it seems that this use case is not well supported. Pogo generated the source code, but created a method
Tango::DevState MyClassName::state()

Indeed a good point. The draft of the device RFC already define to have a State and Status attributes. Would it make sense to define what is a State and a Status command in the Command RFC and make mandatory the State and Status command in the Device RFC?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the method which does the check in POGO:
https://github.com/tango-controls/pogo/blob/Pogo-Release-9.6.23/org.tango.pogo.pogo_gui/src/org/tango/pogo/pogo_gui/tools/Utils.java#L404

As I understand this code, only alphanumeric characters are allowed [a-z][A-Z][0-9] and the underscore.
The property name shall start with a letter.
An attribute property (and only an attribute property) can start with an underscore.

So if we follow the rules defined by code in POGO, I think we would get something like that for the property names in abnf:

alphanum = ALPHA / DIGIT 
underscore = %x5F
device_property_name = 1*ALPHA *(alphanum / underscore)
class_property_name = 1*ALPHA *(alphanum / underscore)
free_property_name = 1*ALPHA *(alphanum / underscore)
attribute_property_name = 1*(ALPHA / underscore) *(alphanum / underscore)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if we take into account that the maximum property name length is 255 we would get something like:

alphanum = ALPHA / DIGIT 
underscore = %x5F
device_property_name = 1*1ALPHA 0*254(alphanum / underscore)
class_property_name = 1*1ALPHA 0*254(alphanum / underscore)
free_property_name = 1*1ALPHA 0*254(alphanum / underscore)
attribute_property_name = 1*1(ALPHA / underscore) 0*254(alphanum / underscore)

* The Property Value MUST use the following convention:
``` ABNF
property-value = CHAR / 1*CHAR CR
```

TODO: Fix property value which depends on property Tango type, it can be empty and can contain several lines in case of
properties with array types.

### Properties for Device without Database

* A Device started without a Tango Database MAY use a file to persist its Properties. Here is an example file content to show the syntax that must be used in this file:
```
# --- my/tango/device properties
my/tango/device->myProp:value

# This is a comment

CLASS/MyClass->myClassProp: 10
```

TODO: Specify the supported Tango types for Properties (DevBoolean, DevShort, DevUShort, ..., array of DevShort,...)
TODO: Add some precisions about special words like "NaN" and "Inf" and their meaning when using some DevDouble or DevFloat Properties