Skip to content
sebaufro edited this page Feb 20, 2018 · 18 revisions

MQTT integration with ACS monitoring system

The ALMA Common Software (ACS) provides the infrastructure to get values from monitoring points, using the BAsic Control Interface (BACI) specification, which defines a special kind of drivers called DevIO.

A special DevIO was developed to read and write from MQTT channels (link here), although it has been not completely tested, it can provide access to read and write from devices using the MQTT protocol to publish data.

ACS also provides a mechanism to get the data from the BACI properties. The Monitor Collector and the Blobber were implemented as part of this infrastructure to collect data from the monitoring points and then publish the data into a storage respectively.

ALMA has implemented a plugin at Blobber level to push data into a ActiveMQ instance, then a piece a software reads from activeMQ channels and write the data into text files in a filesystem.

Goal

Replicate the monitoring software deployment in a controlled test environment, this shall include integration with MQTT through the BACI DevIO implemntation.

Particular Objectives

  1. Prepare MQTT installation in a standard environment (CentOS 7)
  2. Setup ACS deployment to read from a remote mosquitto broker (outside of ALMA network)
    • Write Baci properties to read from mosquitto using the custom DevIO
    • Setup a CDB (based on XML) to deploy the baci properties and their monitor collector and monitor controller and blobbers.
    • Re-use the ALMA's blobber plugin to push data from blobbers to ActiveMQ
      • Make this plugin available for community usage.
  3. Setup a deployment to read from the baci properties deployed in the previous step and store the date in data filed
    • Install software infrastructure used in ALMA monitoring system in prototype environment
    • Modify the storage namespace to accept data from multiple ACS installations
  4. Bonus: Replace the storage based on files by storage based on influx DB.
    • Write a custom plugin for apache ActiveMQ to write to influx.

Progress report

Week 1

* Installation of Mosquitto MQTT Broker and client Paho c ++ in standard ACS environment (Centos 7)
* Additionally installed in a standard environment in Centos 6.8
* Configuration of the ACS environment (INTROOT, and others)
* Development ACS component to read data from mosquito broker and save them in predefined BACIs properties (only local broker at the moment)
* Added basic channel for writing in SensorTag component
* Local installation for ActiveMQ testing
* Installation activemq-cpp library and test of examples codes simple_producer and simple_async_consumer

Week 2

* Try to understand collector, controller and blobber
* CDB configuration for collector, controller and blobber
* Test remote mqtt broker
* Read broker address from CDB

Week 3

* Join test collector, controller and blobber for SensorTag component.
* Identification dependencies of BlobberPluginAlmaImpl
* Remove all innecesary code from archive
* Compilation and test for BlobberPluginAlmaImpl
* Test for remote activemq server
* Local installation of InfluxDB

Week 4

* Test with python for send data to influxDB.
* Join test collector, controller and blobber for SensorTag component in remote activemq server with properties.config archive.
* Install and configure influxdb in a VM
* Try to compile WS Monitoring (three modules failed)
* Try to compile ICD to solve WS Monitoring dependencies (Many modules failed, but when compiling again WS Monitoring only one module fail)

Week 5

* Successful compilation of WS Monitoring (you need to install libmemcached-devel) 
* Added class in BlobberPluginAlmaImpl to put data directly in remote influxdb 
* Run TMCDistributor and testing of the whole process

Week 6

* Some modifications code, (the installation is now in $ INTROOT, modified spring library, etc)
* Try to understand TMCTArchiver
* Modify the storage namespace to accept data from multiple ACS installations
* Run TMCDistributor and testing of the whole process with two ACS installations

Week 7

* Added custom plugin to write to influxdb
Clone this wiki locally