Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Pull request Compare This branch is 1 commit ahead of andrewmathies:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
adapters/ThunderBoardAdapter
code
data
deployments
devices
edges
portals
roles
triggers
users
.gitignore
Readme.md
bleChart.png
deploy.json
package.json
system.json

Readme.md

alt text

ipm package: thunderboard-visualization

Overview

This package has been successfully deployed to both a Raspberry Pi and an NXP imx6 modular gateway. Whichever edge you use searches for and connects to any nearby Thunderboards via BLE. The gateway gathers data from the Thunderboards, and then forwards it to the platform. The platform then displays the data in a graph in the AnomalyDetection portal.

This is an ipm package, which contains one or more reusable assets within the ipm Community. The 'package.json' in this repo is a ipm spec's package.json, here, which is a superset of npm's package.json spec, here.

Browse ipm Packages

What this system does:

Setup

  1. Flash the firmware of your Thunderboard so the advertisement period for connecting via BLE lasts indefinitely. See the ClearBlade Adapters ThunderBoard Sense Firmware Config pdf for reference.
  2. Create a system on the ClearBlade platform and install this IPM.
  3. Go to the Users tab and create a User. Make sure you can remember your password.
  4. Go to the Adapters tab, select the ThunderBoardAdapter adapter, click the pencil edit icon next to Configuration, and download the pythonScanner.py file. Open it up and change the credentials at the start of the file. Leave the platformURL alone, you can find the System Key and System Secret on the platform if you select the info tab, and then the System Settings subtab. The username and password will be the same as the User you just created. After you've changed the credentials, save the file, go back to the edit configuration modal in the Adapters tab, and replace the python scanner file.
  5. Go to the Edges tab, click on the gear icon next to the already created edge named ThunderboardOnPi. Select Setup Instructions, and change the target dropdown to Linux 32bit-ARM. Don't close this.
  6. SSH into your gateway and navigate to the directory you want the edge and adapter software to be in. Go back to your browser, and in the modal you didn't close from the last instrcution, press the copy button next to the Download textbox. Go back to your terminal, paste the instruction, and run it. Do this for Download, Unzip, Install, Permission. You will also need to do this for the command at the very bottom (starts with edge, should have a copy button), BUT you need to put nohup in front of that command so it will log all of it's output in a nohup.out file instead of your terminal.
  7. Go back to your ClearBlade system, and go to the Adapters tab. Select the ThunderBoardAdapter adapter, select the ThunderboardOnPi edge, and press the play button.
  8. Congratualtions, you're done! Wait for a minute or two and go to the devices tab. If your gateway found a Thunderboard it should have created a device there. You can also check the Messages tab, if you don't see any MQTT messages with thunderboard in the topic name then something went wrong.

Usage

To see a visualization of the data, go to the Portals tab and click on the AnomalyDetection portal. Change the topic field on the right to be thunderboard/environment/<THUNDERBOARD_ID>/_platform (the ID will be a 5 digit number in pretty much every MQQT topic in the Messages tab), and change the Sensor Key field to one of sound, co2, temperature, voc, battery, light, uv, humidity, pressure.

Once you have setup the system and followed all of the instructions, you should get MQTT messages on the platform several times a minute with data collected from the Thunderboard.

Assets

Code Services

AddThunderboardDevice - Checks if a thunderboard device exists in the device table, if yes then sends authentication message back to gateway.
DetectAnomaly - Detects an anomaly in the last X MQTT Messages
ExampleDetectAnomaly - Runs anomaly detection against an example dataset, returns an array of Anomalies with metadata.
GetRecentAnomalies - This service retrieves the 10 most recent data points from the Anomalies collection
SaveAnomalyConfiguration - Ingests updated anomaly configuration, and saves to AnomalyConfiguration Collection
SendAlert - Fetches the anomaly row, checks for alerting configuration, and sends email, text alerts

Code Libraries

AnomalyDetection - Core library for detecting anomalies in datasets
AnomalyDetectionConstants - Optional credentials for Alerting (SMS via Twilio, Email via Sendgrid)
jStat - Dependency on jstat-statistics-toolkit
SendGridEmail - Dependency on send-grid-email
TwilioSMS - Dependency on twilio-sms-library

Code Triggers

NewThunderboardDevice - triggers on MQQT message published to +/status/+
SendAlertsTrigger - triggers when item created in Anomalies collection

Portals

AnomalyDetection - Portal for viewing and configuring live datafeeds, rules, and alerts

API

Members

SEND_GRID_API_URI

Creator: Robert Reinold Updated: 2017-10-02T00:00:00Z Version: v2.0 Tags: email, sendgrid, marketing, mail, api, rest, http

Usage:

  1. Create a free SendGrid Account.
  2. Log into your SendGrid account, and view the Settings > API Keys tab. Create an API Key with full access to "Mail Send" rights.
  3. Replace <SEND_GRID_API_KEY> with SendGrid API key
  4. Replace <ORIGIN_EMAIL_ADDRESS> with your desired email address. This will be the 'sender' of the email.
  5. Add 'SendGridEmail' as a dependency to your code services (Settings > Requires > Add)
template

Functions

AnomalyDetection()

Detects Anomalies with Moving Median Decomposition See attached whitepaper for Anomaly Detection for Predictive Maintenance https://files.knime.com/sites/default/files/inline-images/Anomaly_Detection_Time_Series_final.pdf

Run a self-training anomaly detection algorithm against a given dataset

Twilio(user, pass, sourceNumber)

Sends a text message using Twilio's REST API.

AddThunderboardDevice(body)

When the gateway detects a thunderboard via BLE, it checks to see if it has configuration values that match a device in the device table. If not it creates a device. This code service triggers when a MQQT message is sent for the above reason, and if there does exist a matching device then it sends a message back to the gateway authenticating the device so it can start gathering data.

DetectAnomaly()AnomalyVisual

Detects an anomaly in the last X MQTT Messages Uses 'AnomalyConfiguration' config to check for a key in the JSON of each message. ex. 'sensor_key' will be set to 'temperature', so we know to pull the temperature key/value pair from a mqtt message like this: {"temperature":40,"humidity":31}

ExampleDetectAnomaly()Array.<Anomaly>

Example logic for detecting anomalies in a dataset of sensor values

SaveAnomalyConfiguration(anomalyConfiguration)

Ingests updated anomaly configuration, and saves to AnomalyConfiguration Collection

SendAlert(item_id)Object

This is triggered by an Anomaly being recorded, and inserted into the Anomalies Collection

  • Fetches the anomaly row
  • Checks for alerting configuration
  • Sends email, text alert

Typedefs

Calibration
Anomaly
AnomalyVisual : Object
self-explanatorydata

This service retrieves the 10 most recent data points from the Anomalies collection.

SEND_GRID_API_URI

Creator: Robert Reinold Updated: 2017-10-02T00:00:00Z Version: v2.0 Tags: email, sendgrid, marketing, mail, api, rest, http

Usage:

  1. Create a free SendGrid Account.
  2. Log into your SendGrid account, and view the Settings > API Keys tab. Create an API Key with full access to "Mail Send" rights.
  3. Replace <SEND_GRID_API_KEY> with SendGrid API key
  4. Replace <ORIGIN_EMAIL_ADDRESS> with your desired email address. This will be the 'sender' of the email.
  5. Add 'SendGridEmail' as a dependency to your code services (Settings > Requires > Add)

Kind: global variable

template

Kind: global variable

AnomalyDetection()

Detects Anomalies with Moving Median Decomposition See attached whitepaper for Anomaly Detection for Predictive Maintenance https://files.knime.com/sites/default/files/inline-images/Anomaly_Detection_Time_Series_final.pdf

Run a self-training anomaly detection algorithm against a given dataset

Kind: global function
Parameter: number[] dataset array of numbers

AnomalyDetection~calibrate()

Use precomputed anomaly detection calibration profile This can be used to speed up performance for real-time anomaly detection

Kind: inner method of AnomalyDetection

AnomalyDetection~detect() ⇒ Array.<Anomaly>

Run the algorithm against the provided dataset.

Note: This is the most computation-heavy method in this library

Kind: inner method of AnomalyDetection
Returns: Array.<Anomaly> - anomalies - the list of anomalies found in dataset
Parameter: number strictnessOverride (optional) Set the strictness of the anomaly detection

AnomalyDetection~getCalibration() ⇒ Calibration

Fetch the computed anomaly detection calibration parameters

Kind: inner method of AnomalyDetection
Returns: Calibration - calibration

Twilio(user, pass, sourceNumber)

Sends a text message using Twilio's REST API.

Kind: global function

Param Type Description
user string Twilio API account ex. "BC218b72987d86855a5adb921370115a20"
pass string Twilio API passcode ex. "4579ac6ba4fae7b452c03c64aeae40e7"
sourceNumber string Origin phone number of text message, ex "(+1 512-713-2783)"

Twilio~sendSMS(text, recipientNumber, callback)

Send SMS message

Kind: inner method of Twilio

Param Type Description
text string text body
recipientNumber string Formatted phone number ex. "(+1 339-987-2816)"
callback callback ex. function(err, data){}

AddThunderboardDevice(body)

When the gateway detects a thunderboard via BLE, it checks to see if it has configuration values that match a device in the device table. This code service triggers when a MQQT message is sent for the above reason, and if there does exist a matching device then it sends a message back to the gateway authenticating the device so it can start gathering data.

Kind: global function

Param Type Description
body Object contains the MQQT status message sent from the gateway
body.status string should always be "New" if it exists
body.deviceId int is the name of the device specified in the adapter, should match the name of a device in the device table

DetectAnomaly() ⇒ AnomalyVisual

Detects an anomaly in the last X MQTT Messages Uses 'AnomalyConfiguration' config to check for a key in the JSON of each message. ex. 'sensor_key' will be set to 'temperature', so we know to pull the temperature key/value pair from a mqtt message like this: {"temperature":40,"humidity":31}

Kind: global function
Returns: AnomalyVisual - data to visualize this anomaly detection

ExampleDetectAnomaly() ⇒ Array.<Anomaly>

Example logic for detecting anomalies in a dataset of sensor values

Kind: global function
Returns: Array.<Anomaly> - anomalies

SaveAnomalyConfiguration(anomalyConfiguration)

Ingests updated anomaly configuration, and saves to AnomalyConfiguration Collection

Kind: global function

Param Type Description
anomalyConfiguration AnomalyConfiguration map of settings to update

SendAlert(item_id) ⇒ Object

This is triggered by an Anomaly being recorded, and inserted into the Anomalies Collection

  • Fetches the anomaly row
  • Checks for alerting configuration
  • Sends email, text alert

Kind: global function
Returns: Object - response - simple response of success or error

Param Type Description
item_id number the id of the anomaly in Anomalies collection to be retrieved

Calibration

Kind: global typedef
Properties

Name Type Description
min number min value within threshold
max number max value within threshold
strictness number how strict the threshold is against the dataset
medians number intermediary dataset representing the moving medians
pointsPerWindow number calibration, number of points per processing window. may be increased for larger datasets
numWindows number calibration metric derived from pointsPerWindow

Anomaly

Kind: global typedef
Properties

Name Type Description
index number index in the provided dataset
value number value of the data point that is detected as an anomaly

AnomalyVisual : Object

Kind: global typedef
Properties

Name Type Description
toViz Array.<VisualPoints> array of points to be displayed in graph in portal
anomalies Array.<Anomaly> these are datapoints considered to be anomalies

self-explanatory ⇒ data

This service retrieves the 10 most recent data points from the Anomalies collection.

Kind: global typedef
Returns: data - response
Properties

Name Type Description
CURRENTPAGE number the results from the colleciton are all from one page
DATA Array.<Object> each Object in the array contains:
above_threshold boolean true if this datapoint is an anomaly
anomaly_timestamp timestamp when the anomaly was detected
item_id number uuid
sensor_key string which sensor this anomaly is from
threshold_max number if the value is higher than this then its an anomaly
threshold_min number if the value is lower than this then its an anomaly
val number the actual value retrieved from the sensor
You can’t perform that action at this time.