An interactive automaton for a domotic system that commands a wireless plug and interact with synthesized voice and automatic Speech recognition. Tested on Raspbian but developed in pure Java. Internally using MaryTTS for Linux. This project requires the MaryTTSManager, the WholeWordSpeechRecognizer, and the PlugsController projects.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin/org/unirender/interaction/automaton Added Led Management classes Oct 29, 2017
distro Added led blinking management during speech recognition Oct 29, 2017
src/org/unirender/interaction/automaton
.classpath first commit Oct 22, 2017
.gitignore Added Led Management classes Oct 29, 2017
.project
README.md Update README.md Oct 30, 2017
capturedaudio.wav Added led blinking management during speech recognition Oct 29, 2017
capturedaudio1.wav Added led blinking management during speech recognition Oct 29, 2017
interactiveAutomaton.xml first commit Oct 22, 2017
process.properties first commit Oct 22, 2017

README.md

InteractiveAutomaton

An interactive automaton for a domotic system that commands a wireless plug and interact with synthesized voice and automatic Speech recognition. Tested on Raspbian but developed in pure Java. Internally using MaryTTS for Linux. This project requires the MaryTTSManager, the WholeWordSpeechRecognizer, and the PlugsController projects.

Software

The distribution is made up of a Linux software (Windows is available too if MaryTTS - Windows is installed) with scripts to stop and start the interactive automaton. The distro folder contains a compiled version of the software with start and stop scripts.

Required Software

Required Hardware

Installation

sudo apt-get update
sudo apt-get install git
mkdir /home/pi/workspace/
cd /home/pi/workspace/
git clone git@github.com:gianpaolocoro/InteractiveAutomaton.git
git clone git@github.com:gianpaolocoro/MaryTTSManager.git
git clone git@github.com:gianpaolocoro/WholeWordAutomaticSpeechRecognizer.git
git clone git@github.com:gianpaolocoro/PlugsController.git
cd /home/pi/workspace/InteractiveAutomaton/distro
./startInteractionAutomaton.sh (wait until you hear the voice)
./stopInteractionAutomaton.sh (to stop the automaton)

Visual Communication via LED Blinking

On Rasberry Pi 3, the green Led blinks three times just after a word is correctly recognized, one time when it is not recognized. Thus, you can communicate even if you don't want to (or can't) hear the voice.

Properties File

The process.properties file allows tuning some parameters, like the sensitivity of the microphone and the accuracy of the Speech recognizer:

address of the remote plug to control

plugIPAddress=192.168.1.125

path to the MaryTTS installation (to change if Windows is to use or the Text-To-Speech engine is installed in another folder)

maryPath=/home/pi/workspace/MaryTTSManager/marytts-5.2

voice to use

maryVoice=istc-lucia-hsmm

hotword to activate the assistant (choose alternative words from /home/pi/workspace/WholeWordSpeechRecognizer/MODELS/IT/WORDS/)

hotword=assistente

hotword to confirm the activation of the plug

activationHotword=si

Speech recognition score thresholds for hotword and activatio word. Increasing towards 0 means more rigid recognition by the automaton, i.e. word should be uttered clearer and clearer

thresholdHotword=-8800 thresholdActivationWord=-7500

sensitivity of the microphone (0,100) - higher is less sensitive

sensitivity=50

maximum silence (in sec) while expecting a word to be uttered

maxSilence=20

maximum silence (in sec) to wait after a word is uttered

maxSilenceAfterWord=-0.3

maximum time length (in sec) of an uttered word

maxSpeech=10

tolerance of a word: it is allowed to be in the set of the first nBest recognized words

nBest=9

path to the Speech recognition models

modelsPath=/home/pi/workspace/WholeWordSpeechRecognizer/MODELS/

use bluetooth speakers (experimental)

useBlueToothAudio=false