Messaging stack on-boarding

This documentation is designed as a walk-through to onboard users to the ABC-RP messaging stack. It will demonstrate an exemplary onboarding process as well as linking to further reading regarding the standards employed.

Efforts have been made to link to rather than duplicate other documentation.

Overview

The ABC-RP messaging stack is based on the following projects and standards:

• MQTT (Network protocol)

• Building Device Naming Standards initiative (BDNS) (Device naming)

• DBO Data Fields (Point naming)

• DBO Units (Units)

• UDMI (Payload format)

Pre-requisites

Command line tools

The usual command line tools:

• mqtt - aka MQTT CLI

Concepts

This section explains some key concepts of the ABC-RP messaging infrastructure.

We align with the Building Device Naming Standards initiative (BDNS), Digital Buildings Ontology (DBO) and Universal Device Management Interface (UDMI) projects so please review these projects documentation alongside this.

Applications

Applications scope all elements that belong together. Like devices, data, and integrations.

Applications are isolated from each other.

Devices

Each device is owned by exactly one application and has at least one ID to identify it. Device IDs are unique in the scope of an application.

Glossary

• Channel – The channel is an indicator of the device to which data stream the event belongs. Depending on the protocol, this can be set by the device, or generated by the protocol endpoint. For example: In UDMI these would map to pointset or system.

• Device – A device or entity, connecting to the system. Might also be a gateway device, communicating on behalf of other devices (edge node). Might also be another service, providing connectivity for either devices (e.g. The Things Network, Sigfox).

• Gateway device – A device (for example a non IP enabled device) might use another device for sending data to the cloud. So there is one device the data originates from, and (optionally a different) device the connection originates from. If the data is coming from a different device then the connection, the latter device is called a gateway device.

• User – A person accessing the system. Identified by some kind of credentials. Might also be another service. Alternative terms: "account".

• Application – A construct to isolate, scope devices into a group. Data and configuration is not shared between different applications. Alternative terms: "(device) scope", "namespace" and "tenant".

• Instance – An instance of the whole deployment, serving multiple applications.

• Cluster – A Kubernetes cluster, possibly running other workload as well.

Endpoint hostnames

End point hostnames will be provided by ABC-RP as part of your onboarding information. Example values are shown below:

• mqtt.facilities.abc-rp.com:443 For devices connecting to MQTT Protocol Endpoint

Encryption

All our endpoint only support encrypted certificate based communication. This is true even when using (Username)/password for authentication.

Typically we will use Let’s Encrypt TLS certificates which get refreshed regularly. As such you should set up your devices to trust the ISRG Root X1 certificate not the one that is currently on our server as this will be replaced.

Note

Ideally you should use your platform’s certificate bundles which should have updates applied automatically. Please see the Let’s Encrypt Certificate Compatibility page for more information. This will mitigate issues that will arise when the current ISRG Root X1 certificates expires in 2035 (as of 2021).

Application ID

You will be provided a Application ID as part of your onboarding information. This is used internally to manage data and data streams into strictly isolated subsets.

Application IDs:

• Must be DNS labels as defined by RFC1123

• This is necessary to ensure that we can use this ID for Kubernetes resources as well

• Also see Kubernetes Docs

An example Application ID might be:

• swansea-prod-iot

Device ID

Building Device Naming Standards initiative (BDNS)

Device naming should follow the emerging BDNS standard. The specification can be found on this Github page. Also follow the current Abbreviations Register.

Note	Check the development branches for the latest abbreviations

All devices should have a Device ID even if their communication is handled by a Device Gateway.

The device should have a prefix using the following format:

<COUNTRY>-<SUBDIVISION>-<BUILDING>

where:

<COUNTRY>-<SUBDIVISION>-<BUILDING> = assigned location code for building, composed of:

• <COUNTRY> = ISO 3166-1 Alpha-2 Country Code

• <SUBDIVISION> = ISO 3166-2:GB

• <BUILDING> = the project specific abbreviation for the building

Example Device IDs:

• GB-CMD-BLD2_TSTAT-1 First thermostat in the UK, Camden, building BLD2.

• GB-SWA-BLD1_TPS-75 Seventy fifth temperature sensor in Swansea, building BLD1

Note	All devices in under an application must have unique names.

Device Gateways

The most straightforward scenario is a device connecting to one of ABC-RP’s protocol endpoints directly via IP based network infrastructure. For this to work, the device needs to use a supported communication protocol. For the ABC-RP stack MQTT is recommended.

In some cases, a device may not be able to directly connect to one of these protocol endpoints. An example is a device that uses a serial bus or radio waves for local communication. Such devices can be connected to a protocol endpoint by means of a device gateway which acts on behalf of the device(s) when communicating with the cloud.

Device Gateways have the BDNS abbreviation CGW. For example:

• GB-CMD-BLD2_CGW-1 First Device Gateway in the UK, Camden, building BLD2.

This gateway, for example, may handle communications for the GB-CMD-BLD2_TSTAT-1 sensor mentioned above.

When creating and modifying Device Gateways they are treated in exactly the same way as Devices.

Credentials

You will be provided a Device credentials as part of your onboarding information.

Note	If you intend to onboard many devices we can discuss ways of automating this process.

Point naming

DBO Data Fields

Each of the devices that we have created will publish various data points.

The naming for these data points should refer to the DBO Data Fields.

For a CDS carbon dioxide sensor device we would use the DBO data point name:

• zone_air_co2_concentration_sensor

A full list of available telemetry fields is available on the DBO Github

Enumeration

DBO Enumeration

In some cases a device may have two points with semantically identical meaning (e.g.: if a device has two identical current sensors). In this case, the two sensors must use the same field name. To differentiate them we allow a numeric increment (1 indexed) to be added to the name, e.g.: <field>_1. For example:

• zone_air_co2_concentration_sensor_1

• zone_air_co2_concentration_sensor_2

If the device only has one point of a type no index is required.

Metadata

Metadata is managed on per a application bases. As part of the onboarding process you will typically be provided with a table / spreadsheet to populate with additional meta data.

Section

Additional level and zone information is provided at a per device level under the section column. These values should follow the ISO 19650 naming of levels and zones:

<level>_<zone>

For example if device GB-CAM-BLD2_MTS-1 is located in the ground floor kitchen the section would be:

GF_KITCHEN-1

If device GB-CAM-BLD2_MTS-2 is located in the second first floor bedroom the section would be:

01_BEDROOM-2

Example ISO 19650 zones and levels:

Table 1. Zone

ID	Description
ZZ	Multiple Zones (e.g. all of the GF)
XX	No Zone Applicable (rarely used)
KITCHEN-1	Name of a particular zone (1 indexed)

Note	See zones spreadsheet for currently used domestic zones.

Table 2. Level

ID	Description
ZZ	Multiple Levels
XX	No Level Applicable (rarely used)
GF	Ground Floor
01	Floor 1
02	Floor 2, etc
RF	Roof / Loft
M1	Mezzanine above Level 1
M2	Mezzanine above Level 2
B1	Floor -1
B2	Floor -2

Units

DBO Units (Units)

All units should be one fo the predefined DBO units.

Note	It is expected that present_value is of type NUMERIC to be fully ingested into our infrastructure. This can include objects that can be cast to this type such as INT and BOOL.

Telemetry

UDMI Event Pointset

When we have identified the point names for our device we need to encode them into a json payload.

To do this we will follow the UDMI Schema.

Tip	We provide a CLI tool validate_udmi to check your json payloads.

An example payload for our Jersey temperature sensor GB-SWA-BLD1_TPS-75 using the above point names may look something like the below:

{
  "version": 1,
  "timestamp": "2019-01-17T14:02:29.364Z",
  "points": {
    "zone_air_temperature_sensor": {
      "present_value": 21.245
    },
    "return_air_temperature_sensor": {
      "present_value": 25.753
    }
  }
}

Note	If developing a Device Gateway you would construct a separate payload for each child device that the gateways communicates on behalf of.

Channel

When using the recommended MQTT Endpoint the MQTT topic can be set by the device to indicate which data stream and event belongs to.

For example when the device sends telemetry to the cloud we would expect this channel or topic to be set to pointset.

Gateways

If using a gateway device, it is expected that the channel and child devices ids of the gateway are indicated using the MQTT topic:

<channel>/<device>

For example if GB-CAM-BLD2_CGW-1 is sending a pointset on behalf of GB-CAM-BLD2_TSTAT-1 the MQTT topic would be:

pointset/GB-CAM-BLD2_TSTAT-1

Publishing Data to the MQTT Endpoint

We will be using the UDMI payload schema so will need to serialising timestamps accordingly. To generate such an example timestamps in bash run the following code:

# Generate Z-notation "Zulu" UTC timestamp + ms
$ date --utc +%FT%T.%3NZ

Note	For an example of serialising timestamps in python review pyudmi’s serialise_timestamp function.

Now we have a timestamp we can create a simulated payload for our device and send it over the MQTT protocol:

$ mqtt pub -v -h mqtt.facilities.abc-rp.com -p 443 \
  -u 'GB-SWA-BLD1_TPS-75@swansea-iot' -pw 'my-strong-password' \
  -s -t pointset -m '{
  "version": 1,
  "timestamp": "2019-01-17T14:02:29.364Z",
  "points": {
    "zone_air_temperature_sensor": {
      "present_value": 21.245
    },
    "discharge_air_temperature_sensor": {
      "present_value": 25.753
    }
  }
}'

Overview of above command:

Option	Example	Description
pub		Publish a message
-h	mqtt.facilities.abc-rp.com	The hostname of the message broker
-p	443	The port of the message broker
-u	GB-SWA-BLD1_TPS-75@swansea-prod-iot	The username for authentication <device>@<application>
-pw	my-strong-password	The password for authentication
-s		Use default ssl configuration if no other ssl options are specified
--cafile	dstrootx3.pem	Path to a file containing trusted CA certificates to enable encrypted communication DST Root CA X3
-t	pointset	The topic to publish to which will be used as the channel
-m		The message to publish is UDMI format
-V	3	The mqtt version used by the client (v5 and v3.1.1 protocols supported)

Debugging

Our MQTT endpoint is currently setup for ingress only. As such you should not expect to be able to subscribe to topics on the server.

You should get debugging information from your client regarding communication and security (certificates / authentication). If this is functioning correctly please contact us and we will advise of any other issues for the messages being ingested into our systems.