A deep look into the performance of the Raspberry Pi for parallel processing of speech recognition and object detection
Victor Barr Derin Ozturk
- Overview
- Running Code From Pi
- Purpose
- Software Used and Parts List
- Results
- Future Work
- Installation of Tensorflow
- Tensorflow for Audio Recognition
- Installing OpenCV on Raspberry Pi 3b+
- OpenCV on Raspberry Pi 3b+
- Installing Other Libraries Used for Human Detection
- Motor Control of the Robot
Main goals: Human Tracking – Detect the location of humans and follow/turn towards them Speech Enabled – Voice enabled commands to turn, go forward, and stop Light Tracking – Follow highest intensity of light
Installation of software on pi was done in python virtual environments located in
/home/pi/.virtualenvs/cv
Add software environments to path by doing the following:
cd ~
source .profile
workon cv
Note Tensorflow was accidently not placed in the virtual environment. This virtual environment just adds the software packages to the root directory.
There are three possible Demos to run:
cd /home/pi/liteBark
python robot.py -S
cd /home/pi/liteBark
python robot.py -H
cd /home/pi/liteBark/tensorflowAudio
python wav_trigger_inference.py
wifi: 1 4 3 . 2 1 5 . one zero six . two one one ethernet: 1 6 9 . 2 5 4 . two four seven . two two seven
Understand the performance capabilities of the Raspberry Pi for machine learning applications Can it train models? Build Tensorflow? Run multiple threads? Power Consumption? Overheating?
Tensorflow 1.7
60,000+ .wav file model
“Go”, “Stop”, “Left”, “Right”, “Yes”, “No”, “Up”, “Down”
OpenCV
Python-based Code
Raspberry Pi 3b+
Pi Camera v2
Motor Bridge
Robot Frame
Corsair Void Headset
Detects speech of multiple keywords Can rotate, and actuate motors based on these keywords
Can detect intensities of light from camera Detect human objects and actuate motors for robot to follow human object
Power surge problems with current output of battery Multithread to combine speech recognition and object detection is too performance heavy
Manipulate threads to improve voice recognition and object detection in combined interface Look into better power options for the Raspberry Pi Add additional wheel to front of the robot 3D print pi mount, more solid wires No Ethernet Cable (fully headless)
Below is taking from the following github readme by samjabrahams. A wonderful guide on getting tensorflow installed. https://github.com/samjabrahams/tensorflow-on-raspberry-pi. We need tensorflow 1.7 instead of 1.1. Be sure to get the correct file for the built tensorflow package.
This is the easiest way to get TensorFlow onto your Raspberry Pi 3. Note that currently, the pre-built binary is targeted for Raspberry Pi 3 running Raspbian 8.0 ("Jessie"), so this may or may not work for you. The specific OS release is the following:
Raspbian 8.0 "Jessie"
Release: March 2, 2017
Installed via NOOBS 2.3
First, install the dependencies for TensorFlow:
sudo apt-get update
# For Python 2.7
sudo apt-get install python-pip python-dev
Next, download the wheel file from this repository and install it:
```shell
# For Python 2.7
wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp27-none-linux_armv7l.whl
sudo pip install tensorflow-1.1.0-cp27-none-linux_armv7l.whl
# For Python 3.4
wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
sudo pip3 install tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whlFinally, we need to reinstall the mock library to keep it from throwing an error when we import TensorFlow:
# For Python 2.7
sudo pip uninstall mock
sudo pip install mock
# For Python 3.3+
sudo pip3 uninstall mock
sudo pip3 install mockAnd that should be it!
Instructions on setting up a Docker image to run on Raspberry Pi are being maintained by @romilly here, and a pre-built image is hosted on DockerHub here. Woot!
This section will attempt to maintain a list of remedies for problems that may occur while installing from pip
This wheel was built with Python 2.7, and can't be installed with a version of pip that uses Python 3. If you get the above message, try running the following command instead:
sudo pip2 install tensorflow-1.1.0-cp27-none-linux_armv7l.whl
Vice-versa for trying to install the Python 3 wheel. If you get the error "tensorflow-1.1.0-cp34-cp34m-any.whl is not a supported wheel on this platform.", try this command:
sudo pip3 install tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
Note: the provided binaries are for Python 2.7 and 3.4 only. If you've installed Python 3.5/3.6 from source on your machine, you'll need to either explicitly install these wheels for 3.4, or you'll need to build TensorFlow from source. Once there's an officially supported installation of Python 3.5+, this repo will start including wheels for those versions.
Following the link below to understand how to create a model and labels from wav file data https://www.tensorflow.org/versions/master/tutorials/audio_recognition
We want to stream audio files from an incoming wav file. The tutorial below explains how to do that
Github code: https://github.com/kiranjose/python-tensorflow-speech-recognition
To install OpenCV from source onto the Raspbian Pi 3 on Raspbian use follow the steps found here: https://www.pyimagesearch.com/2016/04/18/install-guide-raspberry-pi-3-raspbian-jessie-opencv-3/. This tutorial also goes over using virtual environments in python which we highly recommend.
Follow this tutorial and the camera.py class for info on this section: https://www.pyimagesearch.com/2015/11/09/pedestrian-detection-opencv/
To use our human detection methods, you must install the imutils package using pip. To do this:
pip install imutilsFollow the code in robot.py from lines 16 down. This is commented inline