Permalink
Browse files

Init.

  • Loading branch information...
Bofu Chen (bafu)
Bofu Chen (bafu) committed Apr 27, 2017
0 parents commit 3528225d283d4c3a7ea4948a50c10cc61e322bfb

Large diffs are not rendered by default.

Oops, something went wrong.
@@ -0,0 +1,80 @@
# BerryNet: Deep Learning Gateway on Raspberry Pi
This project turns Raspberry Pi 3 into an intelligent gateway with deep learning running on it. No internet connection is required, everything is done locally on the Raspberry Pi 3 itself. At DT42, we believe that bringing deep learning to edge devices is the trend towards the future. It not only saves costs of data transmission and storage but also makes devices able to respond according to the events shown in the images or videos without connecting to the cloud.
![Figure 1]()
Figure 1 shows the software architecture of the project, we use Node.js, MQTT and an AI engine to analyze images or video frames with deep learning. So far, there are two supported AI engines, the classification engine and the object detection engine. Figure 2 shows the differences between classification and object detection.
![Figure 2]()
One of the application of this intelligent gateway is to use the camera to monitor the place you care about. For example, Figure 3 shows the analyzed results from the camera hosted in the DT42 office. The frames were captured by the IP camera and they were submitted into the AI engine. The output from the AI engine will be shown in the dashboard. We are working on the Email and IM notification so you can get a notification when there is a dog coming into the meeting area with the next release.
![Figure 3]()
# Installation
```
$ git clone https://github.com/DT42/BerryNet.git
$ cd BerryNet
$ ./configure
```
# Configuration
All the configurations are in `config.js`.
* Choose AI Engine.
* Two types of AI engines currently: object classifier and object detector.
* Configure IP camera's snapshot access interface.
* MQTT topics.
# Dashboard
## Open dashboard on RPi3 (with touch screen)
Open browser and enter the URL:
`http://localhost:8080/index.html#source=dashboard.json`
The default dashboard configuration file will be loaded.
## Open dashboard on browser from any computer
Open browser and enter the URL:
`http://<gateway-ip>:8080/index.html#source=dashboard.json`
Click the data sources, and change MQTT broker's IP address to the gateway's IP.
For more details about dashboard configuration (e.g. how to add widgets), please refer to [freeboard project](https://github.com/Freeboard/freeboard).
# Provide Image Input
To capture an image via Pi camera
```
$ mosquitto_pub -h localhost -t berrynet/event/camera -m snapshot_picam
```
To capture an image via configured IP camera
```
$ mosquitto_pub -h localhost -t berrynet/event/camera -m snapshot_ipcam
```
To provide a local image
```
$ mosquitto_pub -h localhost -t berrynet/event/localImage -m <image_path>
```
# Discussion
Please refer to the [Google Group](https://groups.google.com/a/dt42.io/d/forum/berrynet) for questions, suggestions, or any idea discussion.
@@ -0,0 +1,50 @@
#! /bin/sh
#
# Copyright 2017 DT42
#
# This file is part of BerryNet.
#
# BerryNet is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# BerryNet is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with BerryNet. If not, see <http://www.gnu.org/licenses/>.
help() {
echo "Usage: $0 <start | stop | status | log>"
echo " start : launch dlsystem"
echo " stop : terminate dlsystem"
echo " status : check the state of all services"
echo " log : dump logfiles of all services"
exit 1
}
if [ $# -lt 1 ]
then
help
fi
case $1 in
start | stop | status)
sudo systemctl $1 detection_server.service agent.service broker.service dashboard.service localimg.service camera.service journal.service cleaner.timer
;;
log)
sudo journalctl -x --no-pager -u detection_server.service
sudo journalctl -x --no-pager -u agent.service
sudo journalctl -x --no-pager -u broker.service
sudo journalctl -x --no-pager -u dashboard.service
sudo journalctl -x --no-pager -u localimg.service
sudo journalctl -x --no-pager -u camera.service
sudo journalctl -x --no-pager -u journal.service
sudo journalctl -x --no-pager -u cleaner.timer
;;
*)
help
esac
@@ -0,0 +1,54 @@
// Copyright 2017 DT42
//
// This file is part of BerryNet.
//
// BerryNet is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// BerryNet is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with BerryNet. If not, see <http://www.gnu.org/licenses/>.
'use strict';
const mosca = require('mosca');
// ascoltatore: https://github.com/mcollina/ascoltatori
const ascoltatore = {
type: 'mongo',
url: 'mongodb://localhost:27017/mqtt',
pubsubCollection: 'ascoltatori',
mongo: {}
};
const settings = {
port: 1883,
backend: ascoltatore,
http: {
port: 3000,
bundle: true,
static: './'
}
};
const server = new mosca.Server(settings);
server.on('clientConnected', (client) => {
console.log('client connected', client.id);
});
// fired when a message is received
server.on('published', (packet, client) => {
console.log('Server published. Avoid showing package content here.');
});
// fired when the mqtt server is ready
server.on('ready', () => {
console.log('Mosca server is up and running');
});
@@ -0,0 +1,80 @@
// Copyright 2017 DT42
//
// This file is part of BerryNet.
//
// BerryNet is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// BerryNet is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with BerryNet. If not, see <http://www.gnu.org/licenses/>.
'use strict';
const fs = require('fs');
const mqtt = require('mqtt');
const request = require('request');
const spawnsync = require('child_process').spawnSync;
const config = require('./config');
const broker = config.brokerHost;
const client = mqtt.connect(broker);
const topicActionLog = config.topicActionLog;
const topicActionInference = config.topicActionInference;
const topicEventCamera = config.topicEventCamera;
const topicNotifyEmail = config.topicNotifyEmail;
const camera_uri = config.ipcameraSnapshot;
const snapshot_file = '/tmp/snapshot.jpg';
const camera_cmd = '/usr/bin/raspistill';
const camera_args = ['-vf', '-hf', '-w', '1024', '-h', '768', '-o', snapshot_file];
function log(m) {
client.publish(topicActionLog, m);
console.log(m);
}
client.on('connect', () => {
client.subscribe(topicEventCamera);
log(`camera client: connected to ${broker} successfully.`);
});
client.on('message', (t, m) => {
log(`camera client: on topic ${t}, received message ${m}.`);
const action = m.toString();
if (action == 'snapshot_picam') {
// Take a snapshot from RPi3 camera. The snapshot will be displayed
// on dashboard.
spawnsync(camera_cmd, camera_args);
fs.readFile(snapshot_file, function(err, data) {
if (err) {
log('camera client: cannot get image.');
} else {
log('camera client: publishing image.');
client.publish(topicActionInference, data);
}
});
} else if (action == 'snapshot_ipcam') {
// Take a snapshot from IP camera. The snapshot will be sent to
// configured email address.
request.get(
{uri: camera_uri, encoding: null},
(e, res, body) => {
if (!e && res.statusCode == 200) {
log('camera client: publishing image.')
client.publish(topicActionInference, body);
} else {
log('camera client: cannot get image.')
}
}
);
} else {
log('camera client: unkown action.');
}
});
@@ -0,0 +1,5 @@
#!/bin/bash
TARGETDIR="./inference/image"
rm -rf $TARGETDIR/snapshot*
@@ -0,0 +1,14 @@
var mqtt = require('mqtt')
//var client = mqtt.connect('mqtt://test.mosquitto.org')
var client = mqtt.connect('mqtt://localhost:1883')
client.on('connect', function () {
client.subscribe('presence')
client.publish('presence', 'Hello mqtt')
})
client.on('message', function (topic, message) {
// message is Buffer
console.log(message.toString())
client.end()
})
@@ -0,0 +1,62 @@
// Copyright 2017 DT42
//
// This file is part of BerryNet.
//
// BerryNet is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// BerryNet is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with BerryNet. If not, see <http://www.gnu.org/licenses/>.
'use strict';
const path = require('path');
var config = {};
// system configs
config.projectDir = __dirname;
config.inferenceEngine = 'detector'; // {classifier, detector}
// gateway configs
function padTopicBase(topic) {
return path.join(config.topicBase, topic)
}
config.snapshot = path.join(
config.projectDir,
'dashboard/www/freeboard/snapshot.jpg');
config.brokerHost = 'mqtt://localhost';
config.topicBase = 'berrynet';
config.topicActionLog = padTopicBase('action/log');
config.topicActionInference = padTopicBase('action/inference');
config.topicEventCamera = padTopicBase('event/camera');
config.topicEventLocalImage = padTopicBase('event/localImage');
config.topicNotifyEmail = padTopicBase('notify/email');
config.topicNotifySMS = padTopicBase('notify/sms');
config.topicDashboardLog = padTopicBase('dashboard/log');
config.topicDashboardSnapshot = padTopicBase('dashboard/snapshot');
config.topicDashboardInferenceResult = padTopicBase('dashboard/inferenceResult');
// IP camera
config.ipcameraSnapshot = '';
// email notification
config.senderEmail = 'SENDER_EMAIL';
config.senderPassword = 'SENDER_PASSWORD';
config.receiverEmail = 'RECEIVER_EMAIL';
// for compatibility
config.sender_email = config.senderEmail;
config.sender_password = config.senderPassword;
config.receiver_email = config.receiverEmail;
// make config importable
module.exports = config;
Oops, something went wrong.

0 comments on commit 3528225

Please sign in to comment.