iort_lib
is the ROS component of the IoT-Robotics Integration project. Other major components of the project are available at the links below:
iort_endpoint
- open source firmware for an ESP32 microcontroller to serve as an AWS IoT Core thinginsitu
- open source situational awareness package with extension supportiort_lib
- this repositoryiort_filters
- exampleinsitu
extensions usingiort_lib
to construct real time data overlays
You should go through these four links in order and follow the setup isntructions in README.md
.
The IoT-Robotics Integration project uses AWS to ingest, store, and serve IoT data to robotic clients. The following sections will guide you through the process of creating a database, adding an ingest rule, and setting up an web API gateway. Before doing anything on the AWS web console, MAKE SURE YOU ARE ON THE SAME REGION AS WHEN YOU SET UP THE ENDPOINT. You can easily switch regions using the drop-down menu in the top right corner of the page.
Search for Timestream
on the AWS console and navigate to Databases
in the sidebar and hit Create database
. Select Standard database
, pick a name (e.g. iort
), and confirm. Then navigate to Tables
in the sidebar and hit Create table
. Select the database you just created, pick a table name (e.g. data
), choose 1 day for the memory store retention and 30 days for the magnetic store retention, then confirm. Don't worry about the table schema as the columns will be automatically populated by the IoT Core Rule that we are about to set up.
Search for IoT Core
on the AWS console and navigate to Act>Rules
in the sidebar and hit Create
. Give your new rule a name and a description, and paste the following into the Rule query statement
:
SELECT
data
FROM
'device/+/data'
Make sure that it is using SQL version 2016-03-23. Then, under Set one or more actions
, hit Add action
and select Write message into a Timestream table
. On the Configure action
screen, choose the database and table you just created in the previous section, and copy the following dimensions:
Dimension Name | Dimension Value |
---|---|
dev_uuid | ${topic(2)} |
dev_time | ${time} |
msg_uuid | ${traceid()} |
latency | ${timestamp() * 1000 - time} |
For the Timestamp
section, copy the following:
Value | Unit |
---|---|
${timestamp()} | MILLISECONDS |
Finally, create a new role and give it a meaningful name like timestream-writer
, then confirm the action. Finally, hit Create rule
.
If you have correctly set up everything up to this point, you should be able to navigate to Timestream>Query Editor
and use the following SQL query to look at the ingested data:
SELECT * FROM "iort"."data" WHERE time between ago(15m) and now() ORDER BY time DESC LIMIT 10
// TODO
First, we must create a role to give our Lambda the permission to read from our Timestream database. Go to the IAM Console, pick the Lambda
use case, and hit Next
. Under Attach permissions policies
, search for timestream
and check AmazonTimestreamFullAccess
, then hit Next
. Add tags (optional), then give your new role a meaningful name and description (e.g. timestream-reader
, role for timestream access permissions
).
Then navigate to Lmabda>Functions
and hit Create function
. Choose Author from scratch
and give your function a meaningful name (e.g. iort_lib_query
), and pick the Node.js 14.x
runtime. Then click Change default execution role
, select Use an existing role
, and pick the role you just created from the drop-down menu. If it is not there, wait a few minutes and try again; AWS takes some time to update every now and then. Confirm by clicking Create function
and wait for AWS to generate the function stub.
Copy the following into index.js
under Code Source
:
const AWS = require('aws-sdk');
const timestreamquery = new AWS.TimestreamQuery({ region: 'us-west-2' });
function parseRes(res, idx) {
var qmsg = res.Rows[idx].Data;
var ret = {};
// indexes into qmsg are left-to-right from timestream queries
ret['time'] = parseInt(qmsg[0].ScalarValue);
ret['msgid'] = qmsg[1].ScalarValue;
ret['data'] = JSON.parse(qmsg[2].ScalarValue);
return ret;
}
exports.handler = async (event) => {
try {
const body = event.queryStringParameters;
const roboParams = event.queryStringParameters;
var params = {};
switch (roboParams.op) {
case 'query':
params.QueryString = roboParams.qs;
break;
case 'get':
default:
params.QueryString =
"SELECT dev_time, msg_uuid, measure_value::varchar as data " +
"FROM \"iort\".\"data\" " +
"WHERE dev\_uuid = \'"+roboParams.uuid+"\'" +
"ORDER BY dev_time DESC LIMIT 1";
}
let response = {
statusCode: 200,
body: 'null'
};
try {
const res = await timestreamquery.query(params).promise();
if(res.Rows.length == 1 ){
response = {
statusCode: 200,
body: JSON.stringify(parseRes(res, 0))
};
}
else if (res.Rows.length > 1)
{
let ret = [];
for (let i = 0; i < res.Rows.length; ++i) {
ret[i] = parseRes(res, i);
}
response = {
statusCode: 200,
body: JSON.stringify(ret)
};
}
else
{
response = {
statusCode: 404,
body: 'ERROR: Data for queried sensors not found'
};
}
} catch (e) {
response = {
statusCode: 500,
body: 'ERROR: ' + e
};
}
return response;
} catch (e) {
let response = {
statusCode: 500,
body: 'ERROR: ' + e
};
return response;
}
};
MAKE SURE THAT THE REGION, DATABASE NAME, AND TABLE NAME MATCH THE ONES YOU JUST CREATED in the following code sections:
const timestreamquery = new AWS.TimestreamQuery({ region: 'us-west-2' });
params.QueryString =
"SELECT dev_time, msg_uuid, measure_value::varchar as data " +
"FROM \"iort\".\"data\" " +
"WHERE dev\_uuid = \'"+roboParams.uuid+"\'" +
"ORDER BY dev_time DESC LIMIT 1";
Click Function Overview>Add Trigger
and select API Gateway
. Choose Create an API
from the dropdown menu, then select HTTP API
for API type
and Open
for Security
. Confirm by hitting Add
. Now go back to your lambda function and navigate to Configuration>Triggers
. Expand the API Gateway details and copy the API endpoint
URL. Copy this URL into inc/config.h
on the following declaration:
#define HTTP_ENDPOINT "copy url here"
For this section you will need your Thing's Rest API Endpoint URL (AWS IoT Core page and navigate to Manage>Things>myEndpoint>Interact
) and certificates that you created when setting up iort_endpoint
.
In inc/config.h
copy your Rest API Endpoint URL on the following declaration:
#define MQTT_ENDPOINT "copy url here"
Copy the certificate into iort_lib/certs/certificate.pem.crt
.
Copy the private key into iort_lib/certs/private.pem.key
.
Copy the root CA into iort_lib/certs/aws-root-ca.pem
.
This project depends on Eclipse Paho MQTT C++ Client Library
which is used for MQTT subscription. Follow the instructions to build and install the Paho C Library and build and install the Paho C++ library
The project also depends on Curl for People
which allows us to programmatically query the Timestream database, but this is included as a submodule in this repository.
Clone the repository into your catkin workspace and run catkin build
:
cd ~/catkin_ws/src
git clone --recursive https://github.com/PaperFanz/iort_lib
catkin build iort_lib
You may also choose to install the library to your system folders by running
sudo catkin install iort_lib