Skip to content

Commit

Permalink
OPC UA PubSub Main Technology Features #250
Browse files Browse the repository at this point in the history
  • Loading branch information
@mpostol
mpostol committed Apr 14, 2018
1 parent b2c0928 commit dafd59e
Show file tree
Hide file tree
Showing 5 changed files with 84 additions and 73 deletions.
133 changes: 68 additions & 65 deletions Networking/SemanticData/README.PubSubMTF.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,112 +3,98 @@

# Introduction

Two distinct patterns are used to transfer data between communication parties:
[OPC UA Part 14: PubSub][OPC.UA.PubSub] promotes interoperability of loosely coupled **PubSub Applications**. By design, they often will not even know each other. Their primary relationship is the shared understanding of:

- Connection-oriented - requires a session that has to be established before any data can be sent between sender and receiver.
- Connectionless - the sender may start sending packets (called datagrams) to the destination without any preceding handshake procedure.
- specific semantics of exchanged data,
- the syntax and semantics of messages that include these data, and
- common underlying messages transport layer.

In general speaking the following two distinct patterns are used to transfer data between communicating parties:

- Connection-oriented: requires a session that has to be established before any data can be sent between sender and receiver.
- Connectionless-oriented: the sender may start sending messages (called packets or datagrams) to the destination without any preceding handshake procedure.

Each has its own advantages and disadvantages. In general, the OPC UA is a session centric communication. The session is established by the **OPC UA Clint** that must connect to the **OPC UA Server** before any data can be exchanged between them. In this Client/Server scenario defined by the Services in Part 4, the data flow is bidirectional over the session. The session entities communicate over a secure channel that is created in the underlying communication layer and relies upon it for secure communication. It enables to log-in using user authentication and authorization. More details you can find in the article:

- [OPC Unified Architecture – Main Technological Features][wordpress.OPCUAMTF]

Using the connection-oriented communication pattern it is difficult or even impossible to gather and process data from mobile things (devices), which is one of the **Internet of Things** paradigms. More details you can find in [IoT versus SCADA/DCS Data Acquisition Patterns][wordpress.IoTVersus]. The [OPC.UA.PubSub][OPC.UA.PubSub] specification offers the connectionless approach as an additional option to session based client-server interoperability and is a consistent part of the OPC UA specifications suit. PubSub fully integrates into the existing OPC UA technology but as result of applying the connectionless communication it is easier to implement low power and low-latency communications on local networks.
Using the connection-oriented communication pattern it is difficult or even impossible to gather and process data from mobile things (e.g. smart devices, cigarettes box, drug blister, etc.), which is one of the **Internet of Things** paradigms. More details you can find in [IoT versus SCADA/DCS Data Acquisition Patterns][wordpress.IoTVersus].

The specification claims that intentionally PubSub is based on the **[OPC UA Information Model][CAS.OPCUAIMD]** with the aim of seamless integration into **OPC UA Servers** and **OPC UA Clients**. Unfortunately **OPC UA Information Model** is not used to promote **PubSub Application** interoperability. This concept is only employed to define **Security Key Service** and **Configuration Service** models, which have only indirect impact on the **PubSub Application** interoperability. Nevertheless, the PubSub communication does not require such a role dependency, i.e there is no necessity for **Publisher** or **Subscriber** to be either an **OPC UA Server** or an **OPC UA Client** to participate in the communication.
> The [OPC.UA.PubSub][OPC.UA.PubSub] specification offers the connectionless approach as an additional option to session based client-server interoperability and is a consistent part of the OPC UA specifications suit.
> **As the result it can be recognized as the IoT ready technology.**
# Services
The specification claims that the PubSub integrates into the existing OPC UA technology but as result of applying the connectionless communication it is easier to implement low power and low-latency communications on local networks. Additionally, the specification states that PubSub is based on the **[OPC UA Information Model][CAS.OPCUAIMD]** with the aim of seamless integration into **OPC UA Servers** and **OPC UA Clients**. Unfortunately,**OPC UA Information Model** is not used to promote **PubSub Applications** interoperability. This concept is only employed to define **Security Key Service** and **Configuration Service** models, which have an only indirect impact on the **PubSub Applications** interoperability. Nevertheless, the PubSub communication does not require such a role dependency, i.e there is no necessity for **Publisher** or **Subscriber** to be either an **OPC UA Server** or an **OPC UA Client** to participate in the communication.

## Introduction
# Services

**PubSub Applications** exchange messages formatted as the `NetworkMessage` structure using underlying transport layer. Directly or indirectly the specification defines the following actors:

* **Publisher**: pushes the current process data formatted as a `NetworkMessage` structure to an underlying transport layer.
* **Publisher**: pushes the current process data formatted as the `NetworkMessage` structure to an underlying transport layer.
* **Subscriber**: consumes the process data, which is recovered from the `NetworkMessages` structure polled from the underlying transport layer.
* **Security Key Service** - provides security keys that can be used to sign and encrypt messages.
* **Configuration Tool** - an external application used to remotely configure **PubSub Application**.

The **Publisher** is the actor that pushes `NetworkMessage` structures to an underlying transport layer. It represents a certain data source, for example, a control device, a manufacturing process, a weather station, or a stock exchange. It may be also **OPC UA Client**, **OPC UA Server** or in general any applications that understand the syntax and semantics of the `NetworkMessage` structure.

The **Subscriber** actors are the consumers of `NetworkMessage` structure, which are polled from the underlying transport layer. They may be **OPC UA Client*, **OPC UA Server** or in general any applications that understand the syntax and semantics of the `NetworkMessage` structure.

The procedure and related data of messages exchange between the **Publisher** and **Subscriber** is described in the section *Communication*.

The **Security Key Service** responsibility is to

**Configuration Tool**

## Communication
The **Publisher** is the actor that pushes `NetworkMessage` structures to an underlying transport layer. It represents a certain data source, for example, a control device, a manufacturing process, a weather station or a stock exchange. It may be also **OPC UA Client**, **OPC UA Server** or in general any applications that understand the syntax and semantics of the `NetworkMessage` structure.

## Messages Exchange Procedure
The **Subscriber** actors are the consumers of `NetworkMessage` structures, which are polled from the underlying transport layer. They may be **OPC UA Client**, **OPC UA Server** or in general any applications that understand the syntax and semantics of the `NetworkMessage` structure.

The **PubSub Applications** are decoupled by exchanging messages over an underlying transport layer. It is worth stressing that by design the **PubSub Application** don't expose any API that can be used to transfer process upper layer data over the network, i.e. it is not communication layer. It means that these applications must produce and consume the process data.
A **Security Key Service (SKS)** provides keys for message security that can be used by the
**Publisher** to sign and encrypt `NetworkMessages` and by the **Subscriber** to verify the signature of and decrypt the `NetworkMessages`.

OPC UA PubSub interoperability doesn't depend on any functionality provided by the underlying transport layer called **Message Oriented Middleware**. In general, according to the specification the **Subscriber** and **Publisher** can be interconnected using any transparent messages transport infrastructure providing unreliable service that provides no guarantees for delivery and no protection from duplication. Applying MQTT means that some functionality related to communication reliability, data selection, and distribution is delegated to the MQTT `Server`.
**Publishers** and **Subscribers** may be configurable through vendor-specific engineering tools or using the dedicated configuration **OPC UA Information Model** described in this standard. This model allows a standard **OPC UA Client** based configuration tool to configure a **PubSub Application** connecting to the embedded **OPC UA Server**. Using remote **Configuration Tool** over an **OPC UA Session** does not determine how dynamic the configuration can be. It is worth stressing that the configuration model doesn't provide any definition dedicated to being used for the data bindings configuration.

The Publisher doesn't have any subscriptions management functionality, namely, it offers a communication paradigm called unsolicited notification. When unsolicited notification occurs, a client receives a message that it has never requested.
# Interoperability

Subscriber must use a filtering mechanism to process only Messages it is interested in. Subscribers do not have any subscription management functionality.
The **PubSub Applications** are decoupled by exchanging messages over an underlying transport layer. It is worth stressing that by design the **PubSub Application** don't expose any API that can be used to transfer upper layer data over the network, i.e. it is not communication layer. It means that these applications must produce and/or consume the process data.

## `NetworkMessage` serialization
**PubSub Applications** interoperability doesn't depend on any functionality provided by the underlying transport layer. According to the specification, the **Subscriber** and **Publisher** can be interconnected using any transparent messages transport infrastructure. It defines two groups of solutions:

- A broker-less: the transport layer is the network infrastructure that is able to route datagram-based messages, e.g [UDP][RFC.UDP], ETHERNET.
- A broker-based: the core component of the transport layer is a message broker, e.g. [AMQP][AMQP] or [MQTT][MQTT].

## Transport Mapping
In both cases, one-to-many relationship between **Publisher** and **Subscriber** can be obtained. For UDP multicast messages distribution may be applied to send Internet Protocol (IP) datagrams to a group of interested receivers in a single transmission. For the broker-based transport, all messages are published to specific queues (e.g. topics, nodes) that the broker exposes and **Subscribers** can listen to these queues.

The syntax of the UDP transporting protocol URL doesn't provide information if a multicast group is to be joined.
The specification doesn't define any subscription management services, namely, it offers a communication paradigm called unsolicited notification. When unsolicited notification occurs, a client receives a message that it has never requested. Using broker-less approach the **Subscriber** must use a filtering mechanism to process only messages it is interested in. Applying broker-based approach means that some functionality related to communication reliability, data selection, and distribution is delegated to it.

## **Security Key Service**
The syntax and semantics of the messages exchanged between the **PubSub Application** are described as the `NetworkMessage` structure. Each `NetworkMessage` includes header information (e.g. identification and security data) and one or more `DataSetMessages`. The `DataSetMessages` may be signed and encrypted in accordance with the configured message security. Each `DataSetMessage` contains process data.

## **Configuration Tool**
The `NetworkMessage` structure can be serialized using the following encoding:

# Security
- UADP: optimized binary encoding.
- JSON: text format as defined in [RFC JSON][RFC.JSON].

Communication security is based on encryption and signing of the messages. To protect message the **Publisher** may sign and encrypt **NetworkMessages** and the **Subscriber** verifies the signature end decrypt the **NetworkMessages**. Bot operation must use approprite security keys. The keys are provided by independent sever **Security Key Service** (SKS).
This specification lists the following protocols that have been selected to be used as the underlying transport layer:

# Configuration



# Conclusion

> By design the component `DataManagement` implemented in the project [Networking.SemanticData](https://github.com/mpostol/OPC-UA-OOI/tree/master/Networking/SemanticData) is compliant with the [OPC UA Specification Part 14 PubSub](https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-14-pubsub/).
> It must be stressed that because the [OPC UA Specification Part 14 PubSub](https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-14-pubsub/) has been just released the content of this document may change in future.
- [Advanced Message Queuing Protocol][AMQP]
- [Message Queuing Telemetry Transport ][MQTT]
- [User Datagram Protocol][RFC.UDP]
- Ethernet defined by IEEE 802.3

# Normative References

The following documents, in whole or in part, are normatively referenced in this document and are indispensable for its application.

- [OPC Unified Architecture Specification Part 14: PubSub Release 1.04 February 06, 2018][OPC.UA.PubSub]
- [OPC Unified Architecture Specification Part 7: Profiles Release 1.04][OPC.UA.Profiles]

# Glossary

- PubSub Applications
> Any instance of software program that conforms to the PubSub specification
- **PubSub Applications**

> Any instance of the software program that conforms to the PubSub specification.
- Publisher-subscriber communication pattern

- session

> <session>

> Publish-subscribe is a messages distribution scenario where senders of messages, called publishers, do not send them directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are. In the publish-subscribe model, subscribers typically receive only a subset of the total messages published. The process of selecting messages for reception and processing is called filtering. There are two common forms of filtering: topic-based and content-based.
- Connection-oriented communication pattern

> Requires a session connection (analogous to a phone call) be established before any data can be sent. This method is often called a "reliable" network service. It can guarantee that data will arrive in the same order. Connection-oriented services set up virtual links between end systems through a network, as shown in Figure 1. Note that the packet on the left is assigned the virtual circuit number 01. As it moves through the network, routers quickly send it through virtual circuit 01.
> Data exchange scenario that requires a session connection be established before any data can be sent. Connection-oriented services set up virtual links between applications through a network. The session is responsible to retain a state information or status about each communicating partner for the duration of multiple requests. An OPC UA Client/Server connection is a stateful connection because both systems maintain information about the session itself during its life.
- Communication pattern Does not require a session connection between sender and receiver. The sender simply starts sending packets (called datagrams) to the destination. This service does not have the reliability of the connection-oriented method, but it is useful for periodic burst transfers. Neither system must maintain state information for the systems that they send transmission to or receive transmission from. A connectionless network provides minimal services.
- Connectionless-oriented communication pattern
> Messages exchange scenario that does not require a session connection between sender and receiver. The sender simply starts sending packets (called datagrams) to the destination. Neither system must maintain state information for the systems that they send messages to or receive messages from.

- data polling

> continuous checking of the sensors to see what state they are in, usually in multipoint or multidrop communication (a communication engine with multiple devices attached that share the same line) by sending a message to each device, one at a time, asking each to respond and send new data.
- data subscription

> senders of messages containing the process data fetched by the sensor, called publishers, do not prepare the messages to be sent directly to specific receivers, called subscribers, but instead, they categorize published messages into topics without knowledge of which subscribers, if any, may receive the message. Similarly, subscribers express interest in one or more topics and only receive messages that are of interest, without knowledge of which publishers, if any, there are.
- process data

# References

- [OPC Unified Architecture][wordpress.opc-ua]
Expand All @@ -123,7 +109,6 @@ The following documents, in whole or in part, are normatively referenced in this

[wordpress.OPCUACD]: https://mpostol.wordpress.com/2014/05/08/opc-ua-makes-complex-data-access-possible/


- [OPC Unified Architecture – Main Technological Features][wordpress.OPCUAMTF]

[wordpress.OPCUAMTF]: https://mpostol.wordpress.com/2013/08/04/opc-unified-architecture-main-technological-features/
Expand All @@ -132,22 +117,40 @@ The following documents, in whole or in part, are normatively referenced in this

[CAS.OPCUAIMD]: http://www.commsvr.com/InternetDSL/commserver/P_DowloadCenter/P_Publications/20140301E_DeploymentInformationModel.pdf

- [RFC 768: User Datagram ProtocolJ, 28 August 1980][RFC.UDP]
- [RFC 768: User Datagram ProtocolJ, August 1980][RFC.UDP]

[RFC.UDP]:https://tools.ietf.org/html/rfc768

- [RFC: 791 INTERNET PROTOCOL, September 1981][RFC.UDP.IP]

[RFC.UDP.IP]:https://tools.ietf.org/html/rfc791

- [RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format][RFC.JSON]

[RFC.JSON]:https://tools.ietf.org/html/rfc8259

- [OPC Unified Architecture Specification Part 14: PubSub Release 1.04 February 06, 2018][OPC.UA.PubSub]

[OPC.UA.PubSub]:https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-14-pubsub/
[OPC.UA.PubSub]: https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-14-pubsub/

- [OPC Unified Architecture Specification Part 7: Profiles Release 1.04][OPC.UA.Profiles]
- [OPC Unified Architecture Specification Part 7: Profiles Release 1.04 November 1, 2017][OPC.UA.Profiles]

[OPC.UA.Profiles]:https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-7-profiles/

- [OASIS MQTT Version 3.1.1 specification][MQTT]

[MQTT]:http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html

- [OASIS Advanced Message Queuing Protocol (AMQP) Version 1.0][AMQP]

[AMQP]:http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-overview-v1.0-os.html

[RFC.UDP.IP]:https://tools.ietf.org/html/rfc791


- [OOI.Networking.UDPMessageHandler][OOI.Networking.UDPMessageHandler]

[OOI.Networking.UDPMessageHandler]:https://github.com/mpostol/OPC-UA-OOI/tree/master/Networking/UDPMessageHandler
[OOI.Networking.ReferenceApplication]:https://github.com/mpostol/OPC-UA-OOI/tree/master/Networking/ReferenceApplication

- [OOI.Networking.ReferenceApplication][OOI.Networking.ReferenceApplication]

[OOI.Networking.ReferenceApplication]:https://github.com/mpostol/OPC-UA-OOI/tree/master/Networking/ReferenceApplication

0 comments on commit dafd59e

Please sign in to comment.