Skip to content

Home

Jump to bottom
Priya Pappachan edited this page Nov 27, 2018 · 2 revisions

OpenIoE is an Open-source middleware platform for building, managing, and integrating connected products with the Internet of Everything. It enables you to subscribe to data streams and get data from the sensors and store them. The application was generated using JHipster application generator.

Main Features

  • A high-perfomance Java server side with Spring Boot capable of storing data of IoT devices
  • Message broker support with ActiveMQ Artemis
  • Multi Protocol support (MQTT, AMQP)
  • NoSQL data storage for sensor data using Cassandra
  • Frontend for the IoT server which can show data and manage devices
  • Support for timeline series modelling

Table of Contents

Docker

  • All the services required to run the platform can be easily started via bundled docker-compose.yml located in rest_api/src/main/docker
  • Browse to rest_api/src/main/docker and run docker-compose up -d
  • Open 127.0.0.1 in the web browser where you can configure a user account and create devices and sensors
  • To stop the platform, run docker-compose stop in the same folder as above
  • Logs can be observed via docker-compose logs [CONTAINER] (ex: ioe-app)

Installation

Clone the Github repository

Before doing any actual work, it is necessary to prepare your local machine for development and deploy OpenIoE locally.

  1. Install git on your local computer. If on Ubuntu, run 'sudo apt-get install git-core'

  2. Set up your name and email address in git configuration:

    $ git config --global user.name "username"

    $ git config --global user.email useremail

  3. Go to http://github.com and create a GitHub account, if you don’t have one yet. Use the previously introduced email to create the GitHub account. That way GitHub will be able to associate your git commits with your GitHub profile.

  4. Fork the OpenIoE repo. In the upper left corner there is Fork button. A “copy” of the project will appear in the list of your repositories.

  5. Generate SSH keys and add the public key to your GitHub account. GitHub provides a very good article on how to do that.

  6. Go to the page where the forked OpenIoE project is. Copy the SSH clone URL from the right sidebar. Clone the forked project to your system files:

    $ git clone git@github.com:username/OpenIoE.git

Install and configure cassandra

Prerequistes

  1. The latest version of Java 8, either the Oracle Java Standard Edition 8 or OpenJDK 8. To verify that you have the correct version of java installed, type java -version.
  2. For using cqlsh, the latest version of Python 2.7. To verify that you have the correct version of Python installed, type python --version.

Cassandra installation steps

  1. Add the Apache repository of Cassandra to /etc/apt/sources.list.d/cassandra.sources.list, for example for version 3.6: echo "deb http://www.apache.org/dist/cassandra/debian 36x main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
  2. Add the Apache Cassandra repository keys: curl https://www.apache.org/dist/cassandra/KEYS | sudo apt-key add -
  3. Update the repositories: sudo apt-get update
  4. Install Cassandra: sudo apt-get install cassandra

Configure 'ioe' section in application.yml

The application.yml file can be found in OpenIoE/rest_api/src/main/resources/config. Set cassandraUrl under ioe section to localhost.

Install dependencies

You must install and configure the following dependencies on your machine for running the application.

  1. Install gradle and gradlew. We use gradlew to run spring boot in the backend.
  2. Install node.js. We use Node to run a development web server and build the project. In most cases, you'll also want to also install npm, which is the Node.js package manager.
  3. Install development tools like Bower and BrowserSync.
  4. Install gulp and all gulp dependencies using the commands 'npm install -g gulp' and 'npm install --save-dev'

Concepts

User

A new user can be created in OpenIoE with username, password and email id. The user account has to be activated by the admin in order to login as the new user.

Device

A device is an entity in OpenIoE which can be any computing device that connects wirelessly to a network with the ability to transmit data. The device is associated with a user who controls the device. The devices are embedded with sensors and each device can have many sensors. A name, description, type and serial is also defined for a device. Also there is one to many relationship from device to subscriptions which is defined below.

Sensor

The sensor is another entity in OpenIoE. The sensor is associated with a device and a user. The sensor also has one to many relationship to subscriptions. The sensor can have a storeType as NORMAL or EXPIRE. These are the storeTypes defined in nosql.StoreTypes.java. A name, description, type and serial is also defined for sensors. Only name field is defined as mandatory field. The storeType should be NORMAL in order to avoid errors.

When a new sensor is created successfully, an empty table is created in cassandra automatically for the newly created sensor. This cassandra table has fields data, event time, description and value. Cassandra is used only for handling sensor data.

In order to insert sensor data to Cassandra storage POST operation can be used. The data can be provided in json format. In order to retrieve sensor data from cassandra, rest api can be used. The data can be stored and retrieved using message broker also.

Subscription

A subscription is another entity that can be created in OpenIoE. The subscription is associated with a user, device and sensor with one to many mappings. The subscription has fields id, device, user, sensor and topicFilter where no fields are mandatory.

Publication

A publication is another entity that can be created in OpenIoE. The publication is associated with a user, device and sensor with one to many mappings. The publication has fields id, device, user, sensor and topicFilter where no fields are mandatory.

Sensor data

The sensor data is an entity which can be created in OpenIoE. It is used to store the sensor data using message broker and rest api.

Usage

Make sure Cassandra server is up

You can start Cassandra with 'sudo service cassandra start' and stop it with 'sudo service cassandra stop'. Verify that Cassandra is running by invoking nodetool status from the command line.

Run Artemis message broker

The executable for running the artemis message broker is found in /message_broker/bin. Run ./artemis run in order to start the broker.

Run Spring Boot backend

Spring Boot is run by 'gradlew' executable in /rest_api. Run sudo ./gradlew to start running the application.

Go to http://localhost:8080 in your browser where you can configure a user account and create devices and sensors. While creating sensors, StoreType field value should be NORMAL or EXPIRE in order to avoid Server errors.

Example

Suppose user John has many devices one of which is Moto X. Moto X has various sensors. John can create a new account in OpenIoE by providing username, password and email id. The user account has to activated by admin by signing as admin. A new user entry will be created in the user table along with created by, created date, last modified by and last modified date details. The user john can then sign in to the OpenIoE. When signed in, an entry will be added to Audit Event table with event type AUTHENTICATION_SUCCESS.

Suppose John has user id 1. John can create a new device with following fields.

Device id    : 1
Name         : Moto X(String type)
Description  : device(String type)
Type         : smartphone (String type)
Serial       : device (String type)
User         : 1 (select user from dropdown)

Only name field is mandatory. A device is created with device id 1.

John can now create sensor for MotoX device with the following fields.

Sensor id    : 1
Name         : GPS(String type)
Description  : GPS sensor(String type)
Type         : smartphone sensor (String type)
Serial       : sensor (String type)
StoreType    : NORMAL
Table id     : 1 (String type)
Device       : 1 (select device from dropdown)
User         : 1 (select user from dropdown)

A sensor with sensor id 1 is created. John can create many sensors for the same device. A table of the format data_x where x is an integer will be created in cassandra. Let the table created be data_1.

A subscription can be created by John for a sensor in a device created by a user. No fields are mandatory. The user has to create a new subscription in order to subscribe to required topics of sensors.

Subscription id  : 1
Device           : 2 (select device from dropdown)
Sensor           : 2 (select sensor from dropdown)
User             : 2 (select user from dropdown)
TopicFilter      : gps(String type)

The user has to create a new publication in order to publish to topics. A publication can be created by John for a sensor in a device created by a user.

Publication id   : 1
Device           : 2 (select device from dropdown)
Sensor           : 2 (select sensor from dropdown)
User             : 2 (select user from dropdown)
TopicFilter      : gps(String type)

When new data is received using http post method from the devices, OpenIoE checks if this sensor and topic has a publication entry. If it is found then data is published to the topic specified in the message.

Publish - subscribe behavior

The sensor data can be published using rest end point and message broker with MQTT support. Both methods accept the payload as JSON. An example for publishing data with rest endpoint using curl command is

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Authorization: Bearer <access_token>' -d '{
  "data": "24",
  "description": "test",
  "sensorId": 1,
  "timestamp": "2017-03-19T09:47:44.526Z",
  "topic" : "gps"
}' 'http://127.0.0.1:8080/api/sensors/{id}'

The user can also publish data using MQTT protocol with MQTT client applications by sending the data of the following form to the topic 'gps'.

  "data": "24",
  "description": "test",
  "sensorId": 1,
  "timestamp": "2017-03-19T09:47:44.526Z"

The user can subscribe to topics defined for subscription entity using MQTT client applications by providing the topic name as gps. Whenever a new data is published, it will be stored in Cassandra and can be received by the user.

Storing data

Data can be stored in Cassandra via the REST endpoint and the message broker. Both methods accept the payload as JSON. An example is

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Authorization: Bearer <access_token>' -d '{
  "data": "24",
  "description": "test",
  "sensorId": 1,
  "timestamp": "2017-03-19T09:47:44.526Z"
}' 'http://127.0.0.1:8080/api/sensor-data'

The request should pass the Authentication token. The authentication token can be obtained either from ioe api documentation page - http://127.0.0.1:8080/#/docs or from the browser's localStorage under the key jhi-authenticationToken.

Retreiving data

Using the REST end point data can be retreived. http://127.0.0.1:8080/api/sensors/1/data where 1 is the sensor id http://127.0.0.1:8080/api/sensors/1/data?dates=2016-07-30,2016-07-2 where multiple dates can be specified. In order to retrieve sensor data of sensor 4 from cassandra storage, following command can be used.

curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer 'http://127.0.0.1:8080/api/sensors/1/data'

Clone this wiki locally