Skip to content

Latest commit

 

History

History
1020 lines (874 loc) · 76.8 KB

Sparkplug_5_Operational_Behavior.adoc

File metadata and controls

1020 lines (874 loc) · 76.8 KB
Raw

Operational Behavior

An MQTT based SCADA system is unique in that the Primary Host Application is not responsible for establishing and maintaining connections to the Edge Nodes as is the case in most existing legacy poll/response device protocols. With an MQTT based architecture, both the Host Applications as well as the Edge Nodes establish MQTT Sessions with one or more central MQTT Servers. This is the desired functionality as it provides the necessary decoupling from any one application and any given Edge Node/Device. Additional Sparkplug Host Application MQTT clients can connect and subscribe to any of the real time data without impacting the Primary Host Application.

Due to the nature of real time SCADA solutions, it is very important for the Primary Host Application and all connected Edge Nodes to have the MQTT Session state information for each other. In order to accomplish this the Sparkplug Topic Namespace definitions for Birth/Death Certificates along with the defined payloads provide both state and context between the Primary Host Application and the associated Edge Nodes. In most use cases and solution scenarios there are two main reasons for this "designation" of a Primary Host Application:

  1. Only the Primary Host Application should have the permission to issue commands to Edge Nodes.

  2. In high availability and redundancy use cases where multiple MQTT Servers are used, Sparkplug Edge Nodes need to be aware of whether the Primary Host Application is connected to each MQTT Server in the infrastructure. If the Primary Host Application STATE shows that an Edge Node is connected to an MQTT Server that the Primary Host Application is NOT connected to, then the Edge Node should connect to the next available MQTT Server where STATE for the Primary Host Application shows 'online=true'.

Timestamps in Sparkplug

An important aspect of Sparkplug is its use of time. All timestamps must be in Coordinated Universal Time (UTC). In order to ensure this is the case, all Sparkplug Edge Nodes and Sparkplug Host Applications must have an accurate mechanism for ensuring their clocks remain accurate. This is typically left to the system operating system using technologies such as Network Time Protocol (NTP). Regardless of the mechanism used, ensuring all timestamps are accurate and in UTC is critical to all timestamps in Sparkplug.

Case Sensitivity in Sparkplug

The MQTT specification states that MQTT topics are case sensitive [MQTTV5-4.7.3]. For example, the topic a/b is different than A/b. Sparkplug in turn is also case sensitive with regard to both topics as well as metric names. So, a metric 'temperature' is not the same as the metric 'Temperature'. However, this generally should be avoided. Many Host Applications may not be able to differentiate the two metric names as unique. Many databases are case-insensitive and would not be able to handle this situation well.

  • [tck-id-case-sensitivity-sparkplug-ids] Edge Nodes in a Sparkplug environment SHOULD NOT have Sparkplug IDs (Group, Edge Node, or Device IDs) that when converted to lower case match

    • For example there should not be two different Edge Nodes publishing NBIRTH messages on these two topics spBv1.0/Group1/NBIRTH/EdgeNode1 and spBv1.0/group1/NBIRTH/edgenode1

  • [tck-id-case-sensitivity-metric-names] An Edge Node SHOULD NOT publish metric names that when converted to all lower case match.

    • For example a DBIRTH should not contain a metric 'a' and another metric 'A'.

Host Application Session Establishment

The Sparkplug Host Application upon startup or reconnect will immediately try to create a Host MQTT Session with the configured MQTT Server infrastructure. Note that the establishment of an Host Application MQTT session is asynchronous of any other MQTT Client session. If Edge Nodes are already connected to the MQTT Server infrastructure, the Sparkplug Host Application will synchronize using the STATE MQTT topic. If associated Edge Nodes are not connected, the Sparkplug Host Application will synchronize with the Edge Nodes and their data streams when the Edge Nodes publish their Birth Certificates. Any Edge Node that has specified this Sparkplug Host Application as its Primary Host Application will will wait to publish their Birth Certificates until after they receive the STATE message denoting that the Primary Host application is online.

Figure 4 - Host Session Establishment

plantuml::assets/plantuml/host-session-establishment.puml[format=svg, alt="Host Session Establishment"]

The session diagram in Figure 3 - Host Session Establishment shows a very simple topology with a single MQTT Server. The steps outlined in the session diagram are defined as follows:

  1. Sparkplug Host Applications will try to create an MQTT Session using the MQTT CONNECT Control Packet. A Death Certificate is constructed into the MQTT Will Topic and Will Payload of the CONNECT Control Packet with a Will QoS set to 1 and Will Retain flag set to true.

    [tck-id-message-flow-phid-sparkplug-clean-session-311] The CONNECT Control Packet for all Sparkplug Host Applications when using MQTT 3.1.1 MUST set the MQTT 'Clean Session' flag to true.

    [tck-id-message-flow-phid-sparkplug-clean-session-50] The CONNECT Control Packet for all Sparkplug Host Applications when using MQTT 5.0 MUST set the the MQTT 'Clean Start' flag to true and the 'Session Expiry Interval' to 0.

    The MQTT CONNECT Control Packet is acknowledged as successful with a valid MQTT CONNACK Control Packet from the MQTT Server. From this point forward in time, the MQTT Server is ready to deliver a Host Death Certificate any time the Sparkplug Host Application MQTT Client loses connectivity to the MQTT Server.

  2. With the MQTT Session established, Sparkplug Host Application MUST issue an MQTT subscription for the defined Sparkplug Topic Namespace.

    [tck-id-message-flow-phid-sparkplug-subscription] The subscription on the Sparkplug Topic Namespace and the STATE topic MUST be done immediately after successfully establishing the MQTT session and before publishing its own STATE message.

  3. [tck-id-message-flow-phid-sparkplug-state-publish] Once an MQTT Session has been established, the Sparkplug Host Application subscriptions on the Sparkplug Topic Namespace have been established and the STATE topic subscription has been established, the Sparkplug Host Application MUST publish a new STATE message.

    [tck-id-message-flow-phid-sparkplug-state-publish-payload] The Host Application Birth Certificate Payload MUST be JSON UTF-8 data. It MUST include two key/value pairs where one key MUST be 'online' and its value is a boolean 'true'. The other key MUST be 'timestamp' and the value MUST be the same value set in the immediately prior MQTT CONNECT packet’s Will Message payload.

    [tck-id-message-flow-phid-sparkplug-state-publish-payload-timestamp] The timestamp value in the Host Application Birth Certificate payload MUST be the same value set in the immediately prior MQTT CONNECT packet’s Will Message payload.

    The Host Application is now ready to start receiving MQTT messages from any connected Edge Node within the infrastructure. At this point, the Host Application can update the MQTT Client metrics in the Host Application with a current state of 'online' once each Edge Node publishes its Sparkplug NBIRTH and DBIRTH messages. Since the Sparkplug Host Application is also relying on the MQTT Session to the MQTT Server(s), the availability of MQTT Servers to the Host Application is also being monitored and reflected in the MQTT Client metrics in the Host Application.

  4. If at any point in time Host Application loses connectivity with the defined MQTT Server(s), the online=true state of the Server is immediately reflected in the MQTT Client metrics in the Host Application.

    All metric data associated with any Sparkplug Edge Node that was connected to that MQTT Server and known by the Host Application MUST be updated to a STALE data quality if the Host Application loses connection to the MQTT Server.

[tck-id-message-flow-hid-sparkplug-state-message-delivered] After publishing its own Host Application STATE message, if at any point the Host Application is delivered a STATE message on its own Host Application ID with a 'online' value of false, it MUST immediately republish its STATE message to the same MQTT Server with a 'online' value of true and the 'timestamp' set to the same value that was used for the timestamp in its own prior MQTT CONNECT packet Will Message payload.

Edge Node Session Establishment

Prior to sending an NBIRTH message, the MQTT client associated with the Edge Node must subscribe to receive NCMD messages with the following rules.

  • [tck-id-message-flow-edge-node-ncmd-subscribe] The MQTT client associated with the Edge Node MUST subscribe to a topic of the form 'spBv1.0/group_id/NCMD/edge_node_id' where group_id is the Sparkplug Group ID and the edge_node_id is the Sparkplug Edge Node ID for this Edge Node. It MUST subscribe on this topic with a QoS of 1.

    • This subscription is mandatory as Edge Nodes MUST be able to respond to 'rebirth requests'.

After subscribing, the Edge Node must follow these additional rules.

  • [tck-id-message-flow-edge-node-birth-publish-connect] Any Edge Node in the MQTT infrastructure MUST establish an MQTT Session prior to publishing NBIRTH and DBIRTH messages.

  • [tck-id-message-flow-edge-node-birth-publish-will-message] When a Sparkplug Edge Node sends its MQTT CONNECT packet, it MUST include a Will Message.

  • [tck-id-message-flow-edge-node-birth-publish-will-message-topic] The Edge Node’s MQTT Will Message’s topic MUST be of the form 'spBv1.0/group_id/NDEATH/edge_node_id' where group_id is the Sparkplug Group ID and the edge_node_id is the Sparkplug Edge Node ID for this Edge Node

  • [tck-id-message-flow-edge-node-birth-publish-will-message-payload] The Edge Node’s MQTT Will Message’s payload MUST be a Sparkplug Google Protobuf encoded payload.

  • [tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq] The Edge Node’s MQTT Will Message’s payload MUST include a metric with the name of 'bdSeq', the datatype of INT64, and the value MUST be incremented by one from the value in the previous MQTT CONNECT packet unless the value would be greater than 255. If in the previous NBIRTH a value of 255 was sent, the next MQTT Connect packet Will Message payload bdSeq number value MUST have a value of 0.

  • [tck-id-message-flow-edge-node-birth-publish-will-message-qos] The Edge Node’s MQTT Will Message’s MQTT QoS MUST be 1.

  • [tck-id-message-flow-edge-node-birth-publish-will-message-will-retained] The Edge Node’s MQTT Will Message’s retained flag MUST be set to false.

Edge Nodes can be configured to support the concept of a 'Primary Host Application'. In this case, the Edge Node must wait until the Primary Host Application is online and subscribed to Sparkplug messages before the Edge Node publishes its NBIRTH and DBIRTH messages. Specifying a Primary Host is not required for an Edge Node. But it is often desired. For example say an Edge Node is in a Sparkplug environment and there is a single consuming Host Application that historizes the data. It would not be beneficial for the Sparkplug Edge Node to publish data if the Host Application is not connected and subscribed to messages. Instead, it would be better for the Edge Node to store data while the Host Application is offline. Once the Host Application is properly connected, it could then send all of its stored data and continue publishing normally. Once the Sparkplug Edge Node has successfully connected to the MQTT Server, it must publish a NBIRTH message. The NBIRTH message must follow the following rules. Note if Primary Host is configured for the Edge Node, it must also wait until the Primary Host denotes it is online before the Edge Node publishes its NBIRTH message.

  • [tck-id-message-flow-edge-node-birth-publish-phid-wait] If the Edge Node is configured to wait for a Primary Host Application it MUST verify the Primary Host Application is online via the STATE topic before publishing NBIRTH and DBIRTH messages.

    • [tck-id-message-flow-edge-node-birth-publish-phid-wait-id] If the Edge Node is configured to wait for a Primary Host Application it MUST validate the Host Application ID as the last token in the STATE message topic string matches the configured Primary Host Application ID for this Edge Node.

    • [tck-id-message-flow-edge-node-birth-publish-phid-wait-online] If the Edge Node is configured to wait for a Primary Host Application it MUST validate the 'online' boolean flag is true in the STATE message payload before considering the Primary Host Application to be online and active.

    • [tck-id-message-flow-edge-node-birth-publish-phid-wait-timestamp] If the Edge Node is configured to wait for a Primary Host Application it MUST validate the timestamp value is greater than or equal to the previous STATE message timestamp value in the STATE message payload before considering the Primary Host Application to be online and active. If no previous STATE message timestamp value has been received by this Edge Node it MUST consider the incoming STATE message to be the latest/valid.

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-topic] The Edge Node’s NBIRTH MQTT topic MUST be of the form 'spBv1.0/group_id/NBIRTH/edge_node_id' where group_id is the Sparkplug Group ID and the edge_node_id is the Sparkplug Edge Node ID for this Edge Node

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-payload] The Edge Node’s NBIRTH payload MUST be a Sparkplug Google Protobuf encoded payload.

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-payload-bdSeq] The Edge Node’s NBIRTH payload MUST include a metric with the name of 'bdSeq' the datatype of INT64 and the value MUST be the same as the previous MQTT CONNECT packet.

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-qos] The Edge Node’s NBIRTH MQTT QoS MUST be 0.

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-retained] The Edge Node’s NBIRTH retained flag MUST be set to false.

  • [tck-id-message-flow-edge-node-birth-publish-nbirth-payload-seq] The Edge Node’s NBIRTH payload MUST include a 'seq' number that is between 0 and 255 (inclusive).

    • This will become the starting sequence number which all following messages will include a sequence number that is one more than the previous up to 255 where it wraps back to zero.

  • [tck-id-message-flow-edge-node-birth-publish-phid-offline] If the Edge Node is configured to wait for a Primary Host Application, it is connected to the MQTT Server, and receives a STATE message on its configured Primary Host, the timestamp value in the payload is greater than or equal to the timestamp value included in the prior 'online' STATE message, and the 'online' value is false, it MUST immediately publish an NDEATH message and disconnect from the MQTT Server and start the connection establishment process over.

    • If the Edge Node did not previously receive a STATE message from this Primary Host Application, it can not check the timestamp value against the previous value. In this case it MUST honor the 'online' boolean status flag as denoted in the payload.

Most implementations of a Sparkplug Edge Node for real time SCADA systems will try to maintain a persistent MQTT Session with the MQTT Server Infrastructure. But there are use cases where the MQTT Session does not need to be persistent. In either case, an Edge Node can try to establish an MQTT Session at any time and is completely asynchronous from any other MQTT Client in the infrastructure. The only exception to this rule is the use case where there are multiple MQTT Servers and a Primary Host Application. Note this does not refer to the use of the MQTT 'Clean Session' flag in MQTT 3.1.1 or the 'Clean Start' flag in MQTT 5.0. All types of MQTT clients (both Host and Edge Nodes) in a Sparkplug system MUST always set the 'Clean Session' flag in the MQTT 3.1.1 CONNECT packet to true. When using MQTT 5.0 the 'Clean Start' flag must be set to true and the MQTT 'Session Expiry Interval' to zero.

Figure 5 - Edge Node MQTT Session Establishment

plantuml::assets/plantuml/edge-node-mqtt-session-establishment.puml[format=svg, alt="Edge Node MQTT Session Establishment"]

The session diagram in Figure 4 - Edge Node MQTT Session Establishment shows a very simple topology with a single MQTT Server. The steps outlined in the session diagram are defined as follows:

  1. The Edge Node MQTT Client will attempt to create an MQTT connection to the available MQTT Server(s) using the MQTT CONNECT Control Packet. The Death Certificate constructed into the Will Topic and Will Payload follows the format defined in section on NDEATH messages.

  2. The subscription to NCMD level topics ensures that Edge Node targeted messages from the Primary Host Application are delivered. The subscription to DCMD ensures that device targeted messages from the Primary Host Application are delivered. In infrastructures with multiple MQTT Servers and a designated Primary Host Application, the subscription to STATE informs the Edge Node the current state of the Primary Host Application. At this point the Edge Node has fully completed the steps required for establishing a valid MQTT Session with the Primary Host Application.

  3. Once an MQTT Session has been established, the Edge Node MQTT client MUST publish an application level NBIRTH as defined here. At this point, the Primary Host Application will have all the information required to build out the Edge Node metric structure and show the Edge Node in an 'online' state once it publishes its NBIRTH and DBIRTH messages.

  4. If at any point in time the Edge Node MQTT Client loses connectivity to the defined MQTT Server(s), a Death Certificate (NDEATH) is issued by the MQTT Server on behalf of the Edge Node. Upon receipt of the Death Certificate with a bdSeq number metric that matches the preceding bdSeq number in the NBIRTH messages, the Primary Host Application should set the state of the Edge Node to ‘online=false’ and update all metric timestamps related to this Edge Node. Any defined metrics will be set to a STALE data quality.

    1. The bdSeq number is used to correlate an NBIRTH with a NDEATH. Because the NDEATH is included in the MQTT CONNECT packet, its timestamp (if included) is not useful to Sparkplug Host Applications. Instead, a bdSeq number must be included as a metric in the payload of the NDEATH. The same bdSeq number metric value must also be included in the NBIRTH message published immediately after the MQTT CONNECT. This allows Host Applications to know that a NDEATH matches a specific NBIRTH message. This is required because timing with Will Messages may result in NDEATH messages arriving after a new/next NBIRTH message. The bdSeq number allows Host Applications to know when it must consider the Edge Node offline.

Edge Node Session Termination

[tck-id-operational-behavior-edge-node-intentional-disconnect-ndeath] When an Edge Node disconnects intentionally, it MUST publish an NDEATH before terminating the connection.

[tck-id-operational-behavior-edge-node-intentional-disconnect-packet] Immediately following the NDEATH publish, a DISCONNECT packet MAY be sent to the MQTT Server.

  • If an MQTT DISCONNECT packet is sent by the Edge Node, this signals to the MQTT Server that the Will Message MUST not be delivered by the MQTT Server to subscribers of that message. These subscribers are typically Sparkplug Host Applications. This is why a Death message MUST be published before disconnecting from the MQTT Server. It ensures Edge Nodes are notified the Edge Node is now offline.

  • If an MQTT DISCONNECT packet is not sent by the Sparkplug Edge Node, the MQTT Server will eventually deliver the Will Message (Death Certificate) to the subscribers. However, this can take some time to occur based on when the MQTT Server detects that the Edge Node is no longer connected. By sending the Death Certificate before disconnecting without sending an MQTT DISCONNECT packet, we are ensuring that a Death message will be delivered to subscribing clients promptly. The fact that a second Death message will arrive when the Will Message is delivered is not significant. This is because the Will Message Death message will contain a bdSeq number that matches the bdSeq number that is published by the Edge Node immediately before the disconnect. Because it has a duplicate bdSeq, the Will Message Death message MUST be ignored by the subscribing Sparkplug Host Application clients.

This allows the MQTT Server to be notified that the Edge Node is offline and as a result the MQTT Will Message of the Edge Node will not be delivered by the MQTT Server to subscribed MQTT clients.

When an Edge Node goes offline by sending its NDEATH or if an MQTT Server delivers an NDEATH on behalf of an Edge Node, it is implied that all of the Edge Node’s associated Devices are also offline. In addition, it is also implied that all metrics in the previous associated NBIRTH and all DBIRTHs in this Sparkplug session under that Edge Node are now STALE.

For the following normative statements it is up to the designers of the Sparkplug Host Application with regard to how they 'mark' the Sparkplug Edge Node or Sparkplug Device as 'offline'. It is also up to the designers of the Sparkplug Host Application on how they 'mark' a metric as STALE. This is an important aspect of Sparkplug in that an NDEATH means the data was accurate at a time, but now that the MQTT session has been lost can no longer be considered current or up to date.

Because an NDEATH may be sent on behalf of an Edge Node by an MQTT Server in the MQTT Will Message, the Sparkplug payload timestamp does not represent the time that the Edge Node actually went offline. As a result, the timestamp associated with NDEATH events must use the timestamp of receipt on the Sparkplug Host Application. This is in part why Sparkplug Edge Nodes and Host Applications must have synced system clocks and all Sparkplug timestamps must be in UTC time.

  • [tck-id-operational-behavior-edge-node-termination-host-action-ndeath-node-offline] Immediately after receiving an NDEATH from an Edge Node, Host Applications MUST mark the Edge Node as offline using the current Host Application’s system UTC time

  • [tck-id-operational-behavior-edge-node-termination-host-action-ndeath-node-tags-stale] Immediately after receiving an NDEATH from an Edge Node, Host Applications MUST mark all metrics that were included in the previous NBIRTH as STALE using the current Host Application’s system UTC time

  • [tck-id-operational-behavior-edge-node-termination-host-action-ndeath-devices-offline] Immediately after receiving an NDEATH from an Edge Node, Host Applications MUST mark all Sparkplug Devices associated with the Edge Node as offline using the current Host Application’s system UTC time

  • [tck-id-operational-behavior-edge-node-termination-host-action-ndeath-devices-tags-stale] Immediately after receiving an NDEATH from an Edge Node, Host Applications MUST mark all of the metrics that were included with associated Sparkplug Device DBIRTH messages as STALEusing the current Host Application’s system UTC time

For the following assertions an 'online STATE message' is one where a Host Application’s JSON payload has the 'online' key’s value set to true. An 'offline STATE message' is one where the Host Application’s JSON payload has the 'online' key’s value set to false.

If the Edge Node is configured to use a Primary Host Application, it must also watch for 'STATE' messages from the Primary Host Application via an MQTT subscription. If the Primary Host Application denotes it is offline, the Edge Node must disconnect from the current MQTT server following these rules:

  • [tck-id-operational-behavior-edge-node-termination-host-offline] If the Edge Node is configured to use a Primary Host Application, it MUST disconnect from the current MQTT Server if the online JSON value is false and the timestamp value is greater than or equal to the previous online STATE message timestamp value.

    • [tck-id-operational-behavior-edge-node-termination-host-offline-reconnect] If the Edge Node disconnects after being in a Sparkplug session due to a valid 'offline STATE message', it MUST attempt to connect to the next MQTT Server in its connection list to start the session establishment procedure over again.

  • [tck-id-operational-behavior-edge-node-termination-host-offline-timestamp] Consider an Edge Node that is configured to use a Primary Host Application and the Edge Node is connected and publishing. Then it receives an offline STATE message. It MUST NOT disconnect if the timestamp value is less than the value from the previous online STATE message.

    • It must not disconnect because the older timestamp value indicates the Host Application MQTT session that is being denoted as lost is not the one the current session the Host Application has established with the MQTT Server. Due to how an MQTT connection can be lost it is possible and likely that an old Host Application death message could be delivered after a new Host Application MQTT session is established. In this case, the timestamp value on the incoming death message will be older than the current timestamp value. For this reason, it must be ignored.

Device Session Establishment

The aim of the Sparkplug Specification is to enable the transport of real time process variable information from existing and new end devices measuring, monitoring, and controlling a physical process into an MQTT infrastructure subsequently a Sparkplug Host Application. In the context of this document an MQTT Device can represent anything from existing legacy poll/response driven PLCs, RTUs, HART Smart Transmitters, etc., to new generation automation and instrumentation devices that can implement a conformant MQTT client natively.

The preceding sections in this document detail how the Sparkplug Host Application interacts with the MQTT Server infrastructure and how that infrastructure interacts with the notion of a Sparkplug Edge Node. But to a large extent the technical requirements of those pieces of the infrastructure have already been provided. For most use cases in this market sector the primary focus will be on the implementation of the Sparkplug Specification between the native device and the Edge Node API’s.

Prior to sending a DBIRTH message, if the Device supports 'writing to outputs' the MQTT client associated with the Sparkplug Device must subscribe to receive DCMD messages with the following rules.

  • [tck-id-message-flow-device-dcmd-subscribe] If the Device supports writing to outputs, the MQTT client associated with the Device MUST subscribe to a topic of the form 'spBv1.0/group_id/DCMD/edge_node_id/device_id' where group_id is the Sparkplug Group ID the edge_node_id is the Sparkplug Edge Node ID and the device_id is the Sparkplug Device ID for this Device. It MUST subscribe on this topic with a QoS of 1.

A Device can publish a DBIRTH as long as an NBIRTH has been sent previously and the MQTT session is active. The DBIRTH message must follow the following rules.

  • [tck-id-message-flow-device-birth-publish-nbirth-wait] The NBIRTH message MUST have been sent within the current MQTT session prior to a DBIRTH being published.

  • [tck-id-message-flow-device-birth-publish-dbirth-topic] The Device’s DBIRTH MQTT topic MUST be of the form 'spBv1.0/group_id/DBIRTH/edge_node_id/device_id' where group_id is the Sparkplug Group ID the edge_node_id is the Sparkplug Edge Node ID and the device_id is the Sparkplug Device ID for this Device.

  • [tck-id-message-flow-device-birth-publish-dbirth-match-edge-node-topic] The Device’s DBIRTH MQTT topic group_id and edge_node_id MUST match the group_id and edge_node_id that were sent in the prior NBIRTH message for the Edge Node this Device is associated with.

  • [tck-id-message-flow-device-birth-publish-dbirth-payload] The Device’s DBIRTH payload MUST be a Sparkplug Google Protobuf encoded payload.

  • [tck-id-message-flow-device-birth-publish-dbirth-qos] The Device’s DBIRTH MQTT QoS MUST be 0.

  • [tck-id-message-flow-device-birth-publish-dbirth-retained] The Device’s DBIRTH retained flag MUST be set to false.

  • [tck-id-message-flow-device-birth-publish-dbirth-payload-seq] The Device’s DBIRTH payload MUST include a 'seq' number that is between 0 and 255 (inclusive) and be one more than was included in the prior Sparkplug message sent from the Edge Node associated with this Device.

In order to expose and populate the metrics from any device, the following simple session diagram outlines the requirements:

Figure 6 - MQTT Device Session Establishment

plantuml::assets/plantuml/mqtt-device-session-establishment.puml[format=svg, alt="MQTT Device Session Establishment"]

The session diagram in Figure 5 - MQTT Device Session Establishment shows a simple topology with all the Sparkplug elements in place i.e. Host Application, MQTT Server(s), Sparkplug Edge Node and this element, the device element. The steps outlined in the session diagram are defined as follows:

This flow diagram assumes that at least one MQTT Server is available and operational within the infrastructure. Without at least a single MQTT Server the remainder of the infrastructure is unavailable.

  1. Assuming MQTT Server is available.

  2. Assuming the Primary Host Application established MQTT Session with the MQTT Server(s).

  3. The Session Establishment of the associated Sparkplug Edge Node is described in Edge Node Session Establishment. This flow diagram assumes that the Edge Node session has already been established with the Primary Host Application. Depending on the target platform, the Edge Node may be a physical "Edge of Network" gateway device polling physical legacy devices via Modbus, AB, DNP3.0, HART, etc, an MQTT enabled sensor or device, or it might be a logical implementation of one of the Eclipse Tahu compatible implementations for prototype Edge Nodes running on a Raspberry Pi. Regardless of the implementation, at some point the device interface will need to provide a state and associated metrics to publish to the MQTT infrastructure.

  4. State #4 in the session diagram represents the state at which the Edge Node is ready to report all of its metric data to the MQTT Server(s) as defined in Sparkplug. It is the responsibility of the Edge Node (logical or physical) to put this information in the form defined in DBIRTH messages. Upon receiving the DBIRTH message, the Primary Host Application can build out the proper metric structure and set the Sparkplug Device to 'online'.

  5. Following the Sparkplug Specification in Device Data Messages (DDATA), all subsequent metrics are published to the Primary Host Application on a Report by Exception (RBE) basis using the DDATA message format. Time based reporting is not explicitly disallowed by the Sparkplug Specification but it is discouraged and often unnecessary.

  6. If at any time the Sparkplug Device cannot provide real time information, the Sparkplug Specification requires that an DDEATH be published. This will inform the Primary Host Application that all metric information associated with that Sparkplug Device be set to a STALE data quality.

Device Session Termination

[tck-id-operational-behavior-device-ddeath] If a Sparkplug Edge Node loses connection with an attached Sparkplug Device, it MUST publish a DDEATH message on behalf of the device.

When a Sparkplug Device goes offline by having its DDEATH published by an Edge Node, it allows Sparkplug Host Applications to know that the Sparkplug Device is no longer reporting current and accurate values to the Edge Node. Therefore the Edge Node is not able to report live/accurate data values on behalf of the Sparkplug Device to the MQTT Server or in turn to Sparkplug Host Applications. As a result the Sparkplug Host Applications must mark the Device as offline and denote the Sparkplug Device’s tags as stale.

For the following normative statements it is up to the designers of the Sparkplug Host Application with regard to how they 'mark' the Sparkplug Device as 'offline'. It is also up to the designers of the Sparkplug Host Application on how they 'mark' a metric as STALE. This is an important aspect of Sparkplug in that an DDEATH means the data was accurate at a time, but now that the connection between the Sparkplug Edge Node and the Sparkplug Device has been lost can no longer be considered current or up to date.

The DDEATH is sent on behalf of a Sparkplug Device by a Sparkplug Edge Node. Because of this, the Sparkplug payload timestamp associated with a DDEATH is considered accurate and must be used as the timestamp for a Sparkplug Device being marked as offline and for its associated metrics being set to STALE.

[tck-id-operational-behavior-edge-node-termination-host-action-ddeath-devices-offline] Immediately after receiving an DDEATH from an Edge Node, Host Applications MUST mark the Sparkplug Device associated with the Edge Node as offline using the timestamp in the DDEATH payload

[tck-id-operational-behavior-edge-node-termination-host-action-ddeath-devices-tags-stale] Immediately after receiving an DDEATH from an Edge Node, Host Applications MUST mark all of the metrics that were included with the associated Sparkplug Device DBIRTH messages as STALE using the timestamp in the DDEATH payload

Sparkplug Host Applications

As noted above, the Sparkplug Host Application has the required permissions to send commands to Edge Nodes and Sparkplug Devices because Edge Nodes need to know that the Primary Host Application is connected to the same MQTT Server that it is connected to or to walk to another server in the infrastructure. Both are common requirements of a mission critical SCADA system.

But unlike legacy SCADA system implementations, all real time process variable information being published through the MQTT infrastructure is available to any number of additional MQTT Clients in the business that might be interested in subsets if not all of the real time data.

The only fundamental difference between a Primary Host Application MQTT Client and other Sparkplug Host Application MQTT Clients is that the Edge Nodes in the infrastructure know to make sure the Primary Host Application is online before publishing data.

Sparkplug Host Application Message Ordering

Sparkplug Host Applications are required to validate the order of messages arriving from Edge Nodes. This is done using the sequence number which is sent in every NBIRTH, DBIRTH, NDATA, and DDATA message that comes from an Edge Node. Because these MQTT messages are sent on different topics, it is possible based on MQTT Server implementations that these messages may arrive at the Sparkplug Host Application in a different order than they were sent from the Edge Node. This can be especially common when using clustered MQTT Servers. It is the responsibility of the Sparkplug Host Application to ensure that all messages arrive within a 'Reorder Timeout'. In typical environments this timeout can be as little as a couple of seconds. In deployments with very slow networks or clustered MQTT servers it may need to be longer. In some environments, the MQTT Server may ensure in-order delivery of QoS0 MQTT messages even across topics. In these cases this timeout could be zero.

For example, if a Sparkplug Host Application receives messages from an Edge Node with sequence numbers 1, 2, and 4 then at the time the message with a sequence number 4 arrives, a timer SHOULD be started within the Host Application. This is the start of the Reordering Timeout timer. A message with sequence number 3 MUST arrive before the Reordering Timeout elapses. If a message with sequence number 3 does not arrive before the timeout, a Rebirth Request should be sent to the Edge Node. This ensures that the session state is properly reestablished. If a message with a sequence number of 3 arrives before the Reorder Timeout occurs then the timer can be shutdown and normal operation of the Host Application can continue.

It is also important to note that depending on the Sparkplug Host Application’s purpose it may make sense to never process messages out of order. It also may make sense to not process a message that arrived out of sequence if its preceding messages didn’t arrive before the Reorder Timeout. These choices are left to the Sparkplug Host Application developer. For example, a Host Application that is a time series database may want to insert all data that arrives regardless of the message order. However, a rules engine Host Application may require that messages are processed in order of their sequence numbers to preserve the order of events as they occurred at the Edge Node.

  • [tck-id-operational-behavior-host-reordering-param] Sparkplug Host Applications SHOULD provide a configurable 'Reorder Timeout' parameter

  • [tck-id-operational-behavior-host-reordering-start] If a Sparkplug Host Application is configured with a 'reordering timeout' parameter and a message arrives with an out of order sequence number, the Host Application MUST start a timer denoting the start of the Reorder Timeout window

  • [tck-id-operational-behavior-host-reordering-rebirth] If a Sparkplug Host Application is configured with a 'reordering timeout' parameter and the Reorder Timeout elapses and the missing message(s) have not been received, the Sparkplug Host Application MUST send an NCMD to the Edge Node with a 'Node Control/Rebirth' request

    • Non-normative comment: In most cases a 'Primary Host Application' would send a Rebirth Request but a Non-Primary Host may not

  • [tck-id-operational-behavior-host-reordering-success] If the missing message(s) that triggered the start of the Reorder Timeout timer arrive before the reordering timer elapses, the timer MUST be terminated and normal operation in the Host Application MUST continue until another out of order message arrives.

Primary Host Application STATE in Multiple MQTT Server Topologies

For implementations with multiple MQTT Servers, there is one additional aspect that needs to be understood and managed properly. When multiple MQTT Servers are available there is the possibility of "stranding" an Edge Node if the Primary command/control of the Primary Host Application loses network connectivity to one of the MQTT Servers. In this instance the Edge Node would stay properly connected to the MQTT Server publishing information not knowing that Primary Host Application was not able to receive the messages.

[tck-id-operational-behavior-primary-application-state-with-multiple-servers-state-subs] When using multiple MQTT Servers and Edge Nodes are configured with a Primary Host Application, the Primary Host Application instance MUST be configured to publish a STATE Birth Certificate and all Edge Nodes configured with a Primary Host Application MUST subscribe to this STATE message.

[tck-id-operational-behavior-primary-application-state-with-multiple-servers-state] Regardless of the number of MQTT Servers in a Sparkplug Infrastructure, every time a Primary Host Application establishes a new MQTT Session with an MQTT Server, the STATE Birth Certificate defined in the STATE description section MUST be the first message that is published after a successful MQTT Session is established with each MQTT Server.

Sparkplug Edge Nodes in an infrastructure that provides multiple MQTT Servers can establish a session to any one of the MQTT Servers.

[tck-id-operational-behavior-primary-application-state-with-multiple-servers-single-server] The Edge Nodes MUST not connected to more than one server at any point in time.

Upon establishing a session, the Edge Node should issue a subscription to the STATE message published by the Primary Host Application. Since the STATE message is published with the MQTT RETAIN flag set, MQTT will guarantee that the last STATE message is always available. The Edge Node should examine the JSON payload of this message to ensure that the value of the 'online' key is true. If the value is false, this indicates the Primary Application has lost its MQTT Session to this particular MQTT Server.

[tck-id-operational-behavior-primary-application-state-with-multiple-servers-walk] If the Primary Host Application is offline as denoted via the STATE MQTT Message, the Edge Node MUST terminate its session with this MQTT Server and move to the next available MQTT Server that is available.

[tck-id-operational-behavior-edge-node-birth-sequence-wait] The Edge Node MUST also wait to publish its BIRTH sequence until an online=true STATE message is received by the Edge Node. This use of the STATE message in this manner ensures that any loss of connectivity between an MQTT Server and the Primary Host Application does not result in Edge Nodes being "stranded" on an MQTT server because of network issues. The following message flow diagram outlines how the STATE message is used when three (3) MQTT Servers are available in the infrastructure:

Figure 7 – Primary Host Application STATE flow diagram

plantuml::assets/plantuml/primary-host-application-state-flow-diagram.puml[format=svg, alt="Primary Host Application STATE flow diagram"]

  1. When an Edge Node is configured with multiple available MQTT Servers in the infrastructure it should issue a subscription to the Primary Host Application STATE message. The Edge Nodes are free to establish an MQTT Session to any of the available servers over any available network at any time and examine the current STATE value. If the STATE message payload is online=false then the Edge Node should disconnect and walk to the next available server.

  2. Upon startup, the configured Primary Host Application’s MQTT Client MUST include the Primary Host Application DEATH Certificate that indicates STATE is online=false with the message RETAIN flag set to true in the MQTT Will Message. Then the Primary Host Application BIRTH Certificate must be published with a STATE payload of online=true. In both of these messages the timestamp value must match each other and represent the current connection time. The timestamp value must be a JSON number and represent the number of UTC milliseconds since Epoch.

  3. As the Edge Node walks its available MQTT Server list, it will establish an MQTT Session with a server that has a STATE message with a JSON payload that has online=true. The Edge Node can stay connected to this server if its MQTT Session stays intact and it does not receive the Primary Host Application DEATH Certificate.

  4. Having a subscription registered to the MQTT Server on the STATE topic will result in any change to the current Primary Host Application STATE being received immediately. In this case, a network disruption causes the Primary Host Application MQTT Session to server #2 to be terminated. This will cause the MQTT Server, on behalf of the now terminated the Primary Host Application MQTT Client, to deliver the Death Certificate to clients that are currently subscribed to it. Upon receipt of the Primary Host Application Death Certificate each Edge Node will disconnect from the current MQTT Server and connect to the next MQTT Server in its list. Before the Edge Node disconnects and connects to the next MQTT Server it must validate that the JSON payload denotes online=false and the timestamp value is greater than or equal to the prior STATE message timestamp value from that Host Application’s BIRTH message.

  5. The Edge Node connects to the next available MQTT Server and since the current STATE on this server is online=true, it can stay connected. In the meantime, the network disruption between the Primary Host Application and MQTT Server #2 has been corrected. The Primary Host Application has a new MQTT Session established to server #2 with an updated Birth Certificate of online=true. Now MQTT Server #2 is ready to accept new Edge Node session requests.

Edge Node NDATA and NCMD Messages

We’ll start this section with a description of how metric information is published to the Primary Host Application from an Edge Node in the MQTT infrastructure. The definition of an Edge Node is generic in that it can represent both physical "Edge of Network Gateway" devices that are interfacing with existing legacy equipment and a logical MQTT endpoint for devices that natively implement the Sparkplug Specification. The NBIRTH Section defines the Edge Node Birth Certificate MQTT Payload and the fact that it can provide any number of metrics that will be exposed in the Primary Host Application. Some examples of these will be "read only" such as:

  • Edge Node Manufacture ID

  • Edge Node Device Type

  • Edge Node Serial Number

  • Edge Node Software Version Number

  • Edge Node Configuration Change Count

  • Edge Node Position (if GPS device is available)

  • Edge Node Cellular RSSI value (if cellular is being used)

  • Edge Node Power Supply voltage level

  • Edge Node Temperature

Other metrics may be dynamic and "read/write" such as:

  • Edge Node Rebirth command to republish all Edge Node and Device Birth Certificates

  • Edge Node Next server command to move to next available MQTT Server

  • Edge Node Reboot command to reboot the Edge Node

  • Edge Node Primary Network (PRI_NETWORK) where 1 = Cellular, 2 = Ethernet

The important point to realize is that the metrics exposed in the Primary Host Application for use in the design of applications are completely determined by what metric information is published in the NBIRTH. This is entirely dependent on the application and use-case. Each specific Edge Node can best determine what data to expose, and how to expose it, and it will automatically appear in the Primary Host Application metric structure. Metrics can even be added dynamically at runtime and with a new NBIRTH and DBIRTH sequence of messages. These metrics will automatically be added to the Primary Host Application metric structure.

The other very important distinction to make here is that Edge Node NDATA and NCMD messages are decoupled from the Sparkplug Device level data and command messages of DDATA and DCMD. This decoupling in the Topic Namespace is important because it allows interaction from all MQTT Clients in the system (to the level of permission and application) with the Edge Nodes, but NOT to the level of sending device commands. The Primary Host Application could provide a configuration parameter that would BLOCK output DDATA and DCMD messages but still allow NDATA and NCMD messages to flow. In this manner, multiple application systems can be connected to the same MQTT infrastructure, but only the ones with DCMD enabled can publish Device commands.

It is also important to note that an Access Control List (ACL) can be used to allow one or more Sparkplug Host Applications to publish NCMD and DCMD messages to one or more Edge Nodes. Furthermore the ability to publish NCMD or DCMD messages by other Sparkplug Host Applications could be blocked. The decoupled nature of the commands and data messages allows for this type of granular access and control.

The following simple message flow diagram demonstrates the messages used to update a changing cellular RSSI value in the Primary Host Application and sending a command from the Primary Host Application to the Edge Node to use a different primary network path.

Figure 8 - Edge Node NDATA and NCMD Message Flow

plantuml::assets/plantuml/edge-node-ndata-and-ncmd-message-flow.puml[format=svg, alt="Edge Node NDATA and NCMD Message Flow"]

  1. Assuming MQTT Server is available.

  2. Assuming the Primary Host Application established MQTT Session with the MQTT Server(s).

  3. The Edge Node has an established MQTT Session and the NBIRTH has been published. Primary Host Application now has all defined metrics and their current value.

  4. The Edge Node is monitoring its local cellular RSSI level. The level has changed and now the Edge Node wants to publish the new value to the associated metric in Primary Host Application.

  5. From an operational requirement, the Edge Node needs to be told to switch its primary network interface from cellular to Ethernet. From the Primary Host Application, the new metric value is published to the Edge Node using a NCMD Sparkplug message.

MQTT Enabled Device Session Establishment

When implementing Sparkplug directly on an I/O enabled Device, there are two options. The notion of a 'Sparkplug Device' can be removed entirely. In this scenario the MQTT Client can publish 'Edge Node level' messages (e.g. NBIRTH, NDEATH, NCMD, and NDATA) and never use the concept of 'Device level' messages (e.g. DBIRTH, DDEATH, DCMD, and DDATA messages. All of the metrics can be published on the Edge Node level Sparkplug verbs and simply omit use of the Device level Sparkplug verbs. Because the Edge Node level verbs encapsulate the MQTT/Sparkplug Session, this is all that is required.

Alternatively, the implementation can use the concept of both Edge Node and Device Sparkplug verbs (NBIRTH, NDEATH, NDATA, NCMD, DBIRTH, DDEATH, DDATA, and DCMD) as any other Gateway based Edge Node would. From any consuming application this would look like any other Edge Node Gateway that may be managing one or more attached devices.

Sparkplug Host Application Session Establishment

Sparkplug Host Applications must follow the following rules when connecting to the MQTT Server.

  • [tck-id-operational-behavior-host-application-host-id] The sparkplug_host_id MUST be unique to all other Sparkplug Host IDs in the infrastructure.

  • [tck-id-operational-behavior-host-application-connect-will] When a Sparkplug Host Application sends its MQTT CONNECT packet, it MUST include a Will Message.

  • [tck-id-operational-behavior-host-application-connect-will-topic] The MQTT Will Message’s topic MUST be of the form 'spBv1.0/STATE/sparkplug_host_id' where host_id is the unique identifier of the Sparkplug Host Application

  • [tck-id-operational-behavior-host-application-connect-will-payload] The Death Certificate Payload MUST be JSON UTF-8 data. It MUST include two key/value pairs where one key MUST be 'online' and it’s value is a boolean 'false'. The other key MUST be 'timestamp' and the value MUST be the same value that was used for the timestamp in its own prior MQTT CONNECT packet Will Message payload.

  • [tck-id-operational-behavior-host-application-connect-will-qos] The MQTT Will Message’s MQTT QoS MUST be 1 (at least once).

  • [tck-id-operational-behavior-host-application-connect-will-retained] The MQTT Will Message’s retained flag MUST be set to true.

Once the Sparkplug Host Application has successfully connected to the MQTT Server, it must publish a birth with the following rules.

  • [tck-id-operational-behavior-host-application-connect-birth] The MQTT Client associated with the Sparkplug Host Application MUST send a birth message immediately after successfully connecting to the MQTT Server.

  • [tck-id-operational-behavior-host-application-connect-birth-topic] The Host Application’s Birth topic MUST be of the form 'spBv1.0/STATE/sparkplug_host_id' where host_id is the unique identifier of the Sparkplug Host Application

  • [tck-id-operational-behavior-host-application-connect-birth-payload] The Birth Certificate Payload MUST be JSON UTF-8 data. It MUST include two key/value pairs where one key MUST be 'online' and it’s value is a boolean 'true'. The other key MUST be 'timestamp' and the value MUST match the timestamp value that was used in the immediately prior MQTT CONNECT packet Will Message payload.

  • [tck-id-operational-behavior-host-application-connect-birth-qos] The Host Application’s Birth MQTT QoS MUST be 1 (at least once).

  • [tck-id-operational-behavior-host-application-connect-birth-retained] The Host Application’s Birth retained flag MUST be set to true.

The following additional rule applies if the Host Application is connecting to more than one MQTT Server.

  • [tck-id-operational-behavior-host-application-multi-server-timestamp] The Host Application MUST maintain a STATE Message timestamp value on a per MQTT Server basis.

    • For example if a connection is lost to one MQTT Server, when the Host Application reconnects and publishes a new STATE message, it must update the STATE Message timestamp for only this MQTT Server and not any others it may be connected to.

Sparkplug Host Application Session Termination

  • [tck-id-operational-behavior-host-application-termination] If the Sparkplug Host Application ever disconnects intentionally, it MUST publish a Death message with the following characteristics.

  • [tck-id-operational-behavior-host-application-death-topic] The Sparkplug Host Application’s Death topic MUST be of the form 'spBv1.0/STATE/sparkplug_host_id' where host_id is the unique identifier of the Sparkplug Host Application.

  • [tck-id-operational-behavior-host-application-death-payload] The Death Certificate Payload registered as the MQTT Will Message in the MQTT CONNECT packet MUST be JSON UTF-8 data. It MUST include two key/value pairs where one key MUST be 'online' and it’s value is a boolean 'false'. The other key MUST be 'timestamp' and the value MUST be a numeric value representing the current UTC time in milliseconds since Epoch.

  • [tck-id-operational-behavior-host-application-death-qos] The Sparkplug Host Application’s Death MQTT QoS MUST be 1 (at least once).

  • [tck-id-operational-behavior-host-application-death-retained] The Sparkplug Host Application’s Death retained flag MUST be set to true.

  • [tck-id-operational-behavior-host-application-disconnect-intentional] In the case of intentionally disconnecting, an MQTT DISCONNECT packet MAY be sent immediately after the Death message is published.

    • If an MQTT DISCONNECT packet is sent by the Host Application, this signals to the MQTT Server that the Will Message MUST not be delivered by the MQTT Server to subscribers of that message. These subscribers are typically Sparkplug Edge Nodes. This is why a Death message MUST be published before disconnecting from the MQTT Server. It ensures Edge Nodes are notified the Host Application is now offline.

    • If an MQTT DISCONNECT packet is not sent by the Sparkplug Host Application, the MQTT Server will eventually deliver the Will Message (Death Certificate) to the subscribers. However, this can take some time to occur based on when the MQTT Server detects that the Host Application is no longer connected. By sending the Death Certificate before disconnecting without sending an MQTT DISCONNECT packet, we are ensuring that a Death message will be delivered to subscribing clients promptly. The fact that a second Death message will arrive when the Will Message is delivered is not significant. This is because the Will Message Death message will contain a timestamp older than the timestamp that is published by the Host Application immediately before the disconnect. Because it has an older timestamp, the Will Message Death message MUST be ignored by the subscribing Sparkplug clients.

Sparkplug Host Application Receive Data

Sparkplug Host Applications are typically designed to receive data from Sparkplug Edge Nodes and optionally write commands back to them. What they do with that data is not specified by the Sparkplug specification. It is left to the implementor of a Sparkplug Host Application to define what they do with the data and what (if anything) they potentially write back to the Edge Nodes via CMD messages. Example Host Applications may use graphical interfaces or dashboards to display Edge Node data. Other Host Applications may insert data into a historical database for later querying. Other Host Applications may perform real-time analytics on the data as it flows from the Sparkplug Edge Nodes.

Because there is so much flexibility in what a Sparkplug Host Application may do with the data it receives there aren’t hard requirements on what it does with it once it receives it. However, there are some things to consider:

  • A Sparkplug Host Application MAY send Node Control/Rebirth NCMD messages if messages arrive out of sequence order and can not be reordered within the sequence reordering timeout. It is often reasonable for whether or not a Host Application sends Rebirths to be a configuration option as this can have an impact on the overall Sparkplug system.

  • A Sparkplug Host Application MAY send Node Control/Rebirth NCMD messages if malformed payloads arrive. Because this can have an impact on the overall system this should be configurable by the Host Application.

  • There are other reasons a Host Application may send out Node Control/Rebirth NCMD messages. These include but are not limited to:

    • Receiving any DBIRTH, NDATA, DDATA, or DDEATH before receiving an NBIRTH from a Sparkplug Edge Node

    • Receiving a metric in an NDATA message that was not included in the previous NBIRTH message

    • Receiving a metric in a DDATA message that was not included in the previous DBIRTH message

    • Receiving an alias value that was not included in the corresponding NBIRTH or DBIRTH

Data Publish

Publishing of data messages occurs from an Edge Node any time it is online as denoted by previously publishing its BIRTH messages within the same MQTT Session. A Sparkplug session begins with an MQTT CONNECT and then the NBIRTH message. A Sparkplug session ends with an NDEATH. Using the fact that MQTT uses TCP as the underlying protocol as well as facilities in Sparkplug to encapsulate a session, data messages are sent 'by exception'. In other words, data only has to be sent when it changes. This is true as long as the session remains established and valid. The following set of rules defines how data messages should be sent.

Rules for Edge Node data (NBIRTH and NDATA) messages:

  • [tck-id-operational-behavior-data-publish-nbirth] NBIRTH messages MUST include all metrics for the specified Edge Node that will ever be published for that Edge Node within the established Sparkplug session.

  • [tck-id-operational-behavior-data-publish-nbirth-values] For each metric in the NBIRTH, the value MUST be set to the current value or if the current value is null, the is_null flag MUST be set to true and MUST NOT have a value specified.

  • [tck-id-operational-behavior-data-publish-nbirth-change] NDATA messages SHOULD only be published when Edge Node level metrics change.

    • In other words, metric values that have not changed within the same Sparkplug Session SHOULD not be resent until a new Sparkplug session is established.

  • NDATA messages SHOULD be aggregated to include multiple metrics.

    • This is up to the application developer in terms of how many metrics should be aggregated in a single message, but it typically doesn’t make sense to publish an MQTT message for every single metric change.

    • Multiple value changes for the same metric MAY be included in the same Sparkplug NDATA message as long as they have different timestamps.

  • [tck-id-operational-behavior-data-publish-nbirth-order] For all metrics where is_historical=false, NBIRTH and NDATA messages MUST keep metric values in chronological order in the list of metrics in the payload.

Rules for Device data (DBIRTH and DDATA) messages:

  • [tck-id-operational-behavior-data-publish-dbirth] DBIRTH messages MUST include all metrics for the specified Device that will ever be published for that Device within the established Sparkplug session.

  • [tck-id-operational-behavior-data-publish-dbirth-values] For each metric in the DBIRTH, the value MUST be set to the current value or if the current value is null, the is_null flag MUST be set to true and MUST NOT have a value specified.

  • [tck-id-operational-behavior-data-publish-dbirth-change] DDATA messages SHOULD only be published when Device level metrics change.

    • In other words, metric values that have not changed within the same Sparkplug Session SHOULD not be resent until a new Sparkplug session is established.

  • DDATA messages SHOULD be aggregated to include multiple metrics.

    • This is up to the application developer in terms of how many metrics should be aggregated in a single message, but it typically doesn’t make sense to publish an MQTT message for every single metric change.

    • Multiple value changes for the same metric MAY be included in the same Sparkplug DDATA message as long as they have different timestamps.

  • [tck-id-operational-behavior-data-publish-dbirth-order] For all metrics where is_historical=false, DBIRTH and DDATA messages MUST keep metric values in chronological order in the list of metrics in the payload.

Commands

Commands are used in Sparkplug to allow Sparkplug Host Applications to send data to Sparkplug Edge Nodes. Examples include writing to outputs of Sparkplug Edge Nodes and Devices or to request Rebirths from Edge Nodes. Custom command endpoints can be declared in an NBIRTH or DBIRTH message by an Edge Node or Device that may support functionality such as rebooting an Edge Node or Device. This is up to the Sparkplug implementor to define what functionality can be exposed.

Security and access is an important aspect of commands. It may be the case that not all Sparkplug Host Applications should have the ability to send commands. This can be controlled in multiple ways. ACLs (Access Control Lists) may be used to allow/disallow certain MQTT clients from publishing NCMD and DCMD messages. Security features in the Sparkplug Host Application itself could be used to allow/disallow certain users or applications from sending certain commands. Security features in the Sparkplug Edge Node application could be used to allow/disallow CMD messages to be honored. There are a number of ways in which this can be achieved based on the use case. However, implementation details are not covered in the Sparkplug Specification and is left to specific application designers to consider.

There are two types of command (CMD) verbs in Sparkplug. These are NCMD and DCMD messages which target Edge Nodes and Devices respectively.

There is one NCMD that is required to be implemented for all Sparkplug Edge Nodes and that is the 'Node Control/Rebirth' command. This exists to allow a Sparkplug Host Application to reset its end-to-end session with a specific Edge Node. For example, say an Edge Node has been in an established Sparkplug session and is publishing DATA messages. Now say a new Sparkplug Host Application connects to the same MQTT Server that the Edge Node is connected to. On the next DATA message published by the Edge Node, the Host Application will receive it without ever having received the BIRTH message(s) associated with the Edge Node. As a result, it can send a 'Rebirth Request' using the 'Node Control/Rebirth' metric to reset its understanding of that Edge Node and become aware of all metrics associated with it.

These are the rules around the 'Node Control/Rebirth' metric.

  • [tck-id-operational-behavior-data-commands-rebirth-name] An NBIRTH message MUST include a metric with a name of 'Node Control/Rebirth'.

  • [tck-id-operational-behavior-data-commands-rebirth-name-aliases] When aliases are being used by an Edge Node an NBIRTH message MUST NOT include an alias for the 'Node Control/Rebirth' metric.

    • This is to ensure that any Host Application connecting to the MQTT Server is capable of requesting a rebirth without knowledge of any potential alias being used for this metric.

  • [tck-id-operational-behavior-data-commands-rebirth-datatype] The 'Node Control/Rebirth' metric in the NBIRTH message MUST have a datatype of 'Boolean'.

  • [tck-id-operational-behavior-data-commands-rebirth-value] The 'Node Control/Rebirth' metric value in the NBIRTH message MUST have a value of false.

A 'Rebirth Request' consists of the following message from a Sparkplug Host Application with the following characteristics.

  • [tck-id-operational-behavior-data-commands-ncmd-rebirth-verb] A Rebirth Request MUST use the NCMD Sparkplug verb.

  • [tck-id-operational-behavior-data-commands-ncmd-rebirth-name] A Rebirth Request MUST include a metric with a name of 'Node Control/Rebirth'.

  • [tck-id-operational-behavior-data-commands-ncmd-rebirth-value] A Rebirth Request MUST include a metric value of true.

Upon receipt of a Rebirth Request, the Edge Node must do the following.

  • [tck-id-operational-behavior-data-commands-rebirth-action-1] When an Edge Node receives a Rebirth Request, it MUST immediately stop sending DATA messages.

  • [tck-id-operational-behavior-data-commands-rebirth-action-2] After an Edge Node stops sending DATA messages, it MUST send a complete BIRTH sequence including the NBIRTH and DBIRTH(s) if applicable.

  • [tck-id-operational-behavior-data-commands-rebirth-action-3] The NBIRTH MUST include the same bdSeq metric with the same value it had included in the Will Message of the previous MQTT CONNECT packet.

    • Because a new MQTT Session is not being established, there is no reason to update the bdSeq number

  • After the new BIRTH sequence is published, the Edge Node may continue sending DATA messages.

Another common use case for sending commands is to use them to 'write' to outputs on Sparkplug Devices. Often these are PLCs or RTUs with writable outputs. NCMD and DCMD messages can be used for these writes. The general flow is for a Host Application to send a command message, the Edge Device receives the message and writes to the output using the native protocol. Then when the output changes value, it results in the Edge Node publishing a DATA message denoting the new value.

For Edge Node level commands, the following rules must be followed.

  • [tck-id-operational-behavior-data-commands-ncmd-verb] An Edge Node level command MUST use the NCMD Sparkplug verb.

  • [tck-id-operational-behavior-data-commands-ncmd-metric-name] An NCMD message SHOULD include a metric name that was included in the associated NBIRTH message for the Edge Node.

    • Sparkplug Edge Node Applications should be resilient to receiving metrics names that were not included in the NBIRTH message.

  • [tck-id-operational-behavior-data-commands-ncmd-metric-value] An NCMD message MUST include a compatible metric value for the metric name that it is writing to.

    • In other words, if the metric has a datatype of a boolean the value must be true or false.

For Device level commands, the following rules must be followed.

  • [tck-id-operational-behavior-data-commands-dcmd-verb] A Device level command MUST use the DCMD Sparkplug verb.

  • [tck-id-operational-behavior-data-commands-dcmd-metric-name] A DCMD message SHOULD include a metric name that was included in the associated DBIRTH message for the Device.

    • Sparkplug Edge Node Applications should be resilient to receiving metrics names that were not included in the DBIRTH message.

  • [tck-id-operational-behavior-data-commands-dcmd-metric-value] A DCMD message MUST include a compatible metric value for the metric name that it is writing to.

    • In other words, if the metric has a datatype of a boolean the value must be true or false.