The Legacy Systems Protocol Converter (LPC), initially developed within the Horizon Europe Interstore project, acts as a middleware, allowing devices that use different communication protocols to exchange data with EMS systems that use the IEEE2030.5 standard. It supports:
- IEEE2030.5 communication: This is the primary function of the Legacy Protocol Converter. It can handle IEEE2030.5 messages in both JSON and XML formats.
- Next-generation NATS messaging: This is a new messaging protocol that the converter uses to communicate with devices and EMS systems. It is designed to be more efficient and scalable than traditional protocols like REST over HTTP.
- MQTT and Modbus protocols: These are common protocols used by many devices. The Legacy Protocol Converter can translate messages from these protocols into the IEEE2030.5 format for use with EMS systems.
Key features of Legacy Protocol Converter:
- Built-in transformation framework: This framework allows users to define how incoming messages should be transformed into the outgoing IEEE2030.5 format. This is important because different devices and systems may use different message formats.
- Configuration file: The converter uses a configuration file to specify connection details for NATS, MQTT, and Modbus devices. Users can also define transformations within the configuration file.
- Flexibility: The converter can support multiple transformations, each with different incoming and outgoing connections, message formats, and structures. This allows for a high degree of flexibility in how the converter is used.
The Legacy Protocol Converter can be deployed and run using:
- A Docker container. Pre-built Docker images are available on Docker Hub, a custom Docker image can be build.
- Using a Java JAR file and execute it on any computer with OpenJDK Java Runtime Environment.
- Compile and build the project out of source code.
- It is possible to integrate LPC in custom projects by including packages.
LPC can be executed on-premise or in the cloud. It supports a variety of environments, including Kubernetes, Docker and classical virtual machines, as well as bare-metal.
Overall, the Legacy Protocol Converter is a tool for enabling communication between devices and EMS systems that use different communication protocols. It supports the latest IEEE2030.5 standard and provides a flexible and efficient way to translate messages between different protocols.
General format of the configuration file is following:
connections:
-
-
registration:
topic:
outgoing-connection:
-
-
message:
transformations:
-
-Each incoming/outgoing connection must be configured in the list of connections so the LPC knows how to connect.
Possible options for each connection are following:
connections:
- name: string
type: NATS/MQTT/RabbitMQ/Modbus
host: string
port: integer
ssl:
default: true/false
username: string
password: string
version: 3/5
virtual-host: string
exchange-name: string
routing-key: string
exchange-type: direct/fanout/topic
reconnect: true/false
device: string
baud-rate: integer
data-bits: integer
parity: none/even/odd/space/mark
stop-bits: integer
...name and type are required keys.
Currently supported parameters for connection with NATS are host, port, username, password and reconnect.
Example of configuration for NATS:
connections:
- name: NATS-connection
type: NATS
host: nats://localhost
port: 4222
username: userTest
password: testUser
reconnect: true
...Currently supported parameters for connection with MQTT are host, port, ssl, version, username, password and reconnect.
Example of configuration for MQTT:
connections:
- name: MQTT-connection
type: MQTT
host: localhost
port: 8883
ssl:
default: true
version: 3
username: username
password: password
reconnect: false
...Currently supported parameters for connection with RabbitMQ are host, port, username, password, virtual-host, exchange-name, routing-key, exchange-type, reconnect.
virtual-host, exchange-name, routing-key, exchange-type are optional.
Default value for virtual-host is /.
Example of configuration for RabbitMQ:
connections:
- name: RabbitMQ-connection
type: RabbitMQ
host: localhost
virtual-host: /
exchange-name: exchange
routing-key: key
exchange-type: direct
...Configuration for Modbus depends on connection type, LPC supports serial connection or TCP connection. For TCP connection parameters host and port are required. For serial connection parameters device is required, optional parameters are baud-rate, data-bits, parity and stop-bits.
Option for RTU over TCP is also supported by configuring both parameters for TCP and serial connection.
Example of configuration for serial Modbus connection:
connections:
- name: Modbus-connection
type: Modbus
device: /dev/ttymxc2
baud-rate: 115200
data-bits: 8
parity: none
...Example of configuration for TCP Modbus connection:
connections:
- name: Modbus-connection
type: Modbus
host: localhost
port: 502
..Registration is optional and provides support for registering the LPC by specified connections. It is designed to send a message once to the specified topic when the LPC is started.
Possible options are:
registration:
topic: string
outgoing-connection:
-
-
message: string- topic: Topic on which the registration message will be sent.
- outgoing-connection: List of connection names, this connections will be used for sending registration message to server. Must match the name in the connections.
- message: Message that will be sent to the specified topic.
Example of registration:
registration:
topic: registration/device/{deviceId}
outgoing-connection:
- NATS-connection
message: '
{
"status": "online"
}
'In the transformations section, each transformation is described, from which topics to listen on to mapping incoming/outgoing messages to specified format and structure.
Possible options for each transformation are following:
connections:
-
-
transformations:
- name: string
description: string
connections:
incoming-connection:
-
-
incoming-topic: string
incoming-format: XML/JSON
outgoing-connection:
-
-
outgoing-topic: string
outgoing-format: XML/JSON
to-outgoing:
retry-count: integer
topic: string
message: string
to-incoming:
retry-count: integer
to-topic: string
message: string
or
to-incoming:
retry-count: integer
modbus-function-code: integer
modbus-device-id: integer
modbus-registers:
- register-address: integer
path: string
type: int8/int16/int32/int64/float32/float64
pattern: string
values: array
interval-request:
interval: integer
request:
modbus-function-code: integer
modbus-device-id: integer
modbus-registers:
- register-address: integer
path: string
type: int8/int16/int32/int64/float32/float64
pattern: string
values: array
or
request:
to-topic: string
reply-from-topic: string
message: stringGeneral options:
- name: Short name of the transformation
- description: Description of the transformation
Connection options:
- connections.incoming-connection: List of connection names, this connections will be used for sending/receiving the data from clients. If using Modbus, only Modbus connections must be listed here. Must match the name in the * connections*.
- connections.incoming-topic: On which topic MQTT/NATS/RabbitMQ client will listen for the incoming messages.
- connections.incoming-format: Format of the incoming messages.
- connections.outgoing-connection: List of connection names, this connections will be used for sending/receiving the data from server. Must match the name in the connections.
- connections.outgoing-topic: On which topic MQTT/NATS/RabbitMQ client will listen for the messages from server.
- connections.outgoing-format: Format of the outgoing messages.
Messages options:
- to-outgoing: Structure of the outgoing message with defined options.
- retry-count: Number of retries for sending the message.
- to-topic: Topic on which the message will be sent.
- message: Message with mappings that will be sent to the specified topic.
- to-incoming: Structure of the incoming message with defined options.
- retry-count: Number of retries for sending the message.
- to-topic: Topic on which the message will be sent.
- message: Message with mappings that will be sent to the specified topic.
- modbus-function-code: Function code for reading/writing data from/to Modbus device.
- modbus-device-id: Client id of the Modbus device.
- modbus-registers: List of definitions of modbus registers used for writing/reading the data.
Interval request options:
- interval-request: Structure of the interval request with defined options.
- interval: Interval in milliseconds for sending the request.
- request: Structure of the request with defined mappings and topic.
- to-topic: Topic on which the message will be sent.
- reply-from-topic: Topic from which the reply will be received.
- message: Message with mappings that will be sent to the specified topic.
- modbus-function-code: Function code for reading/writing data from/to Modbus device.
- modbus-device-id: Client id of the Modbus device.
- modbus-registers: List of definitions of modbus registers used for writing/reading the data.
retry-count is used for all message structures and it specifies the number of retries for sending the message. If not specified, default value is 0.
All topics provide support for placeholders, by using {KEY}.
LPC will replace this KEY with either system variable under the key KEY or with the value from provided Java
argument.
For example, registration/device/{deviceId} will be replaced with registration/device/1 if the deviceId is 1.
Transforming messages from one structure to another is done with the help of a mapper. Each mapper has the following variables that are configurable:
- type
- path
- pattern
- values
type specifies the type to which value must be converted to. Possible values are: integer, float, double, date, datetime and string. When using Modbus, number of bits is required, so integer8 (or int8), int16, int32, int64 and also float32 and float64; int64 is converted to long and float64 is converted to double. date and datetime are converted to long representing the number of milliseconds since January 1, 1970, 00:00:00 GMT.
path specifies path to the value that will be used in new message structure. For XML/JSON this is done using XPath or JSON Pointer (e.g. /OutgoingEvent/currentStatus). For Modbus messages, register address must be provided.
pattern is needed only when type is datetime or date as it specifies the format of the provided temporal value at path.
values is needed only when value must be converted to the index of an array or index to some value from the array.
For easier explanation of options, we will use examples.
We will be transforming JSON structure of IncomingEvent to XML structure of OutgoingEvent. These two structures are used just as examples.
JSON IncomingEvent:
{
"datetime": "28-08-2023 12:00:35",
"status": "active",
"start": "28-08-2023",
"duration": 900
}XML IEEE2030.5 Event:
<Event>
<creationTime>1702909917932</creationTime>
<EventStatus>
<currentStatus>1</currentStatus>
<dateTime>1693216835000</dateTime>
<potentiallySuperseded>false</potentiallySuperseded>
</EventStatus>
<interval>
<duration>900</duration>
<start>1693216835000</start>
</interval>
</Event>connections:
- name: NATS-connection
type: NATS
host: nats://localhost
port: 4222
reconnect: true
- name: MQTT-connection
type: MQTT
ssl:
default: true
host: localhost
port: 8883
username: username
password: password
transformations:
- name: JSON IncomingEvent to XML IEEE2030.5 Event
description: Example showing transformation of messages from JSON to XML
connections:
incoming-connection:
- MQTT-connection
incoming-topic: topic1
incoming-format: JSON
outgoing-connection:
- NATS-connection
outgoing-topic: event/listen
outgoing-format: XML
to-outgoing:
to-topic: event/send
message: '<Event>
<creationTime>$timestamp</creationTime>
<EventStatus>
<currentStatus>
<lpc:mapping>
<path type="integer">/status</path>
<values>["scheduled", "active", "cancelled", "cancelled_with_r", "superseded"]</values>
</lpc:mapping>
</currentStatus>
<dateTime>
<lpc:mapping>
<path type="datetime">datetime</path>
<pattern>dd-MM-yyyy HH:mm:ss</pattern>
</lpc:mapping>
</dateTime>
<potentiallySuperseded>false</potentiallySuperseded>
</EventStatus>
<interval>
<duration>
<lpc:mapping>
<path type="integer">duration</path>
</lpc:mapping>
</duration>
<start>
<lpc:mapping>
<path type="date">/start</path>
<pattern>dd-MM-yyyy</pattern>
</lpc:mapping>
</start>
</interval>
</Event>
'With this transformation we are specifying that the MQTT-connection will subscribe to topic topic1 and LPC will
transform the message to XML
structure, and it will send the transformed message using NATS-connection to topic event/send.
We can see that the mapping is done with the help of XML tag <lpc:mapping>.
For setting the value of OutgoingEvent/currentStatus, we are converting value "active" to integer 1.
Because the IncomingEvent/status is string, and we have provided values options, this means we will map
to the integer based on which index is the value of IncomingEvent/status. So "active" maps to 1.
For setting the value of OutgoingEvent/datetime, we are converting from datetime to long.
Because type is datetime, we need to provide the pattern of the incoming value, so LPC know how to parse the
value, hence dd-MM-yyyy HH:mm:ss. Also note that the leading / is not needed.
Same applies for setting the value of OutgoingEvent/interval/start, but here we are using date.
For setting the value of OutgoingEvent/interval/duration we just need to specify type, because mapping is
done 1 on 1.
There is also reserved keyword $timestamp which is used to set the value of OutgoingEvent/creationTime to
the current time in milliseconds.
For showcasing this we will use the example above but IncomingEvent in XML format and IEEE2030.5 Event in JSON format.
XML IncomingEvent:
<IncomingEvent>
<datetime>28-08-2023 12:00:35</datetime>
<status>active</status>
<start>28-08-2023</start>
<duration>900</duration>
</IncomingEvent>JSON IEEE2030.5 Event:
{
"creationTime": 1702909917932,
"eventStatus": {
"currentStatus": 1,
"dateTime": 1693216835000,
"potentiallySuperseded": false
},
"interval": {
"duration": 900,
"start": 1693216835000
}
}connections:
- name: NATS-connection
type: NATS
host: nats://localhost
port: 4222
reconnect: true
- name: MQTT-connection
type: MQTT
ssl:
default: true
host: localhost
port: 8883
username: username
password: password
transformations:
- name: XML IncomingEvent to JSON IEEE2030.5 Event
description: Example showing transformation of messages from XML to JSON
connections:
incoming-connection:
- MQTT-connection
incoming-topic: topic1
incoming-format: XML
outgoing-connection:
- NATS-connection
outgoing-topic: event/listen
outgoing-format: JSON
to-outgoing:
to-topic: event/send
message: '{
"creationTime": $timestamp,
"eventStatus": {
"currentStatus": {
"lpc:mapping": {
"path": "/IncomingEvent/status",
"type": "integer",
"values": ["scheduled", "active", "cancelled", "cancelled_with_r", "superseded"]
}
},
"dateTime": {
"lpc:mapping": {
"path": "/IncomingEvent/datetime",
"type": "datetime",
"pattern": "dd-MM-yyyy HH:mm:ss"
}
},
"potentiallySuperseded": false
},
"interval": {
"duration": {
"lpc:mapping": {
"path": "/IncomingEvent/duration",
"type": "integer"
}
},
"start": {
"lpc:mapping": {
"path": "/IncomingEvent/start",
"type": "date",
"pattern": "dd-MM-yyyy"
}
}
}
}
'Here we see the mapping is similar as in the example above, but because it is in JSON
format, we are using lpc:mapping key.
For this example, we will use the Modbus connection to read data from the device and transform it to JSON IEEE2030.5 Event.
IncomingEvent structure in Modbus:
- register address 31000: datetime
- register address 31010: status
- register address 31020: start
- register address 31030: duration
JSON IEEE2030.5 Event:
{
"creationTime": 1702909917932,
"eventStatus": {
"currentStatus": 1,
"dateTime": 1693216835000,
"potentiallySuperseded": false
},
"interval": {
"duration": 900,
"start": 1693216835000
}
}connections:
- name: NATS-connection
type: NATS
host: nats://localhost
port: 4222
reconnect: true
- name: Modbus-connection
type: Modbus
device: /dev/ttya
baud-rate: 9600
data-bits: 8
transformations:
- name: XML IncomingEvent to JSON IEEE2030.5 Event
description: Example showing transformation of messages from XML to JSON
connections:
incoming-connection:
- Modbus-connection
outgoing-connection:
- NATS-connection
outgoing-topic: event/listen
outgoing-format: JSON
to-outgoing:
to-topic: event/send
message: '{
"creationTime": $timestamp,
"EventStatus": {
"currentStatus": {
"lpc:mapping": {
"path": "31010",
"type": "int8"
}
},
"dateTime": {
"lpc:mapping": {
"path": "31000",
"type": "int64"
}
},
"potentiallySuperseded": false
},
"interval": {
"duration": {
"lpc:mapping": {
"path": "31030",
"type": "int32"
}
},
"start": {
"lpc:mapping": {
"path": "31020",
"type": "int64"
}
}
}
}
'
interval-request:
interval: 10000
request:
modbus-function-code: 3
modbus-device-id: 1
modbus-registers:
- register-address: 31000
type: int64
- register-address: 31010
type: int8
- register-address: 31020
type: int64
- register-address: 31030
type: int32Here we see that the mapping is done with register addresses, and we are specifying the type of the value at the register address.
Here we have specified that LPC will send a request to the device every 10 seconds to read the data from the registers.
LPC will for each register send new request with function code specified at modbus-function-code to specified
device atmodbus-device-id.
LPC can be deployed as a JAR or as a Docker container.
When deploying, path to the configuration file must be provided either as an argument or mounted as a volume.
Default path to the configuration folder is ./conf.
In this folder, multiple configuration files can be placed, and LPC will read all of them.
Configuration of logging is done with the help of Log4j library. So for configuring logging, one should create file log4j.xml and config logging per Log4j documentation.
Default configuration is following:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration name="kumuluzee">
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="%d %p -- %c -- %marker %m %X %ex %n"/>
</Console>
<File name="file_debug_app" fileName="logs/debug_app.log">
<PatternLayout pattern="%d %p -- %c -- %marker %m %X %ex %n"/>
</File>
<File name="file_info_app" fileName="logs/app.log">
<PatternLayout pattern="%d %p -- %c -- %marker %m %X %ex %n"/>
</File>
<File name="file_lpc" fileName="logs/lpc.log">
<PatternLayout pattern="%d %p -- %c -- %marker %m %X %ex %n"/>
</File>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="console" level="info"/>
<AppenderRef ref="file_info_app" level="info"/>
<AppenderRef ref="file_debug_app" level="debug"/>
</Root>
<Logger name="si.sunesis.interoperability" level="debug">
<AppenderRef ref="file_lpc"/>
</Logger>
</Loggers>
</Configuration>This will print out logs of level INFO or above in the console and in the file app.log.
This will print out logs of level DEBUG or above in the file debug_app.log.
Separate log file for Legacy Protocol Converter is created in order for easier troubleshooting of LPC at lpc.log.
If using different configuration, one should specify the location of the configuration.
Logs are stored in the folder logs in the root of the project. If you are using Docker, logs are stored in the
container. You can also mount the logs folder to the container to store logs on the host machine such as:
docker run -v /path/to/logs:/app/logs -v /path/to/config:/app/conf lpc:latestThis will mount the logs folder /path/to/logs to the container and logs will be available on the host machine.
Build the JAR:
mvn clean packageOPTIONAL If using the custom configuration file for the logging,
then the environment variable with the path to the file must be
set: KUMULUZEE_LOGS_CONFIGFILELOCATION=path/to/file/log4j2.xml
Deploy the application:
java -jar transformation-framework/target/transformation-framework-1.0.jarThis will take the configuration files from ./conf folder. If you want to specify a different folder, you can do
so by providing the path as an argument:
java -DCONFIGURATION=/path/to/config -jar transformation-framework/target/transformation-framework-1.0.jarThis will take the configuration files from /path/to/config folder.
Build the JAR:
mvn clean packageBuild the Docker image:
docker build -t lpc:latest .OPTIONAL If using the custom configuration file for the logging, then this file must be mounted to the container
before running it.
It must be mounted to /app/log-config/log4j2.xml like this:
docker run -v /path/to/log4j2.xml:/app/log-config/log4j2.xml lpc:latestRun the Docker container and mount configuration folder:
docker run -v /path/to/config:/app/conf lpc:latestPre-built Docker images are available here: https://hub.docker.com/r/interstore/legacy-protocol-converter
JRE 17 is required to run the JAR file.
JAR file is located in the official GitHub repository of LPC here: https://github.com/Horizont-Europe-Interstore/Legacy-Protocol-Converter/blob/master/lpc-1.0.jar
You can download the JAR file and then run the following command to start the LPC:
java -jar lpc-1.0.jarThis will take the configuration files from ./conf folder. If you want to specify a different folder, you can do
so by providing the path as an argument:
java -DCONFIGURATION=/path/to/config -jar lpc-1.0.jarThis will take the configuration files from /path/to/config folder.
Docker is required to run the Docker image.
Docker image is located on Docker Hub here: https://hub.docker.com/r/interstore/legacy-protocol-converter
When using Docker to run the LPC, configuration folder must be mounted to the container. You can run the Docker image with the following command:
docker run -v /path/to/config:/app/conf interstore/legacy-protocol-converterThis will mount the configuration folder /path/to/config to the container and LPC will read the configuration files
from that folder.
NATS image is available on Docker Hub: https://hub.docker.com/_/nats
To start NATS server in Docker, you can use the following command:
docker run -d --name nats-main -p 4222:4222 -p 6222:6222 -p 8222:8222 nats:latestThen in order for the LPC to connect to the NATS server, you must configure the two containers to use the same network.
MIT