ghost_bridge

ghost_bridge is a ROS package that connects the Hanson Robotics stack and GHOST. It provides a set of actions that can be called from GHOST and a set of perceptions that are sent to the OpenCog Atomspace, an overview of the actions and perceptions are provided below.

Actions:

• say: make the robot vocalize text.
• gaze-at: turn the robot's eyes towards the given target point.
• face-toward: turn the robot's face towards the given target point.
• blink: set the robot's blink cycle.
• saccade: set the robot's eye saccade cycle, i.e. how the eye's twitch and move around automatically.
• emote: set the robot's emotional state.
• gesture: set a pose on the robot's face.
• soma: sets the robot's background facial expressions.

Perceptions:

• perceive-emotion: perceive an emotion.
• perceive-eye_state: perceive the state of a person's eyes.
• perceive-face-talking: the probability of whether a particular face is talking or not.
• perceive-word: perceive an individual word that is a part of the sentence a person is currently speaking.
• perceive-sentence (ghost): perceive the whole sentence after the user has finished speaking.

Setup

Follow the steps below to setup the ghost_bridge stack.

1. Setup HEAD stack

Setup the Hanson Robotics head stack by following these instructions.

2. Configure ~/.bashrc

Add the following to your ~/.bashrc,

source /opt/ros/kinetic/setup.bash
export HR_WORKSPACE="$(hr env | grep HR_WORKSPACE | cut -d = -f 2)"
source ${HR_WORKSPACE}/HEAD/devel/setup.bash

Ensure you source your ~/.bashrc afterwards:

source ~/.bashrc

3. Install dependencies

3.1 With NVIDIA GPU support

Install the Tensorflow and Dlib GPU dependencies: CUDA Toolkit 9.0 and cuDNN SDK v7. This will also install compatible proprietary NVIDIA GPU drivers. The cuDNN runtime is needed for Tensorflow and the cuDNN developer library is needed to compile Dlib with CUDA support.

To install the CUDA Toolkit 9.0, download:

• CUDA debian network installer

Run the following commands to install CUDA:

cd ~/Downloads
sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda=9.0.176-1

To install cuDNN SDK v7, download (you will need to login, fill in a short questionnaire accept the terms and conditions):

• cuDNN v7.1.4 Runtime Library for Ubuntu16.04 (Deb)
• cuDNN v7.1.4 Developer Library for Ubuntu16.04 (Deb)

Run the following commands to install the cuDNN runtime and developer library:

sudo dpkg -i libcudnn7_7.1.4.18-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.1.4.18-1+cuda9.0_amd64.deb

Add the following to your ~/.bashrc:

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.0/lib64/
export CUDA_BIN_PATH=/usr/local/cuda-9.0

Ensure you source your ~/.bashrc afterwards:

source ~/.bashrc

Install Tensorflow GPU and Keras:

sudo pip install tensorflow-gpu==1.8.0 keras==2.1.6

Install dlib:

cd $HR_WORKSPACE
git clone https://github.com/davisking/dlib.git
cd dlib
sudo -E bash -c "python setup.py install"

3.2 Without NVIDIA GPU support

Install the following:

sudo pip install dlib==19.13.0 tensorflow==1.8.0 keras==2.1.6

4. Checkout ghost repos

Clone ghost_bridge and checkout v0.5.6:

cd ${HR_WORKSPACE}/HEAD/src && git clone https://github.com/opencog/ghost_bridge.git
cd ${HR_WORKSPACE}/HEAD/src/ghost_bridge && git fetch --all --tags --prune
git checkout tags/v0.5.6

Clone ros_people_model:

cd ${HR_WORKSPACE}/HEAD/src && git clone https://github.com/elggem/ros_people_model.git
cd ${HR_WORKSPACE}/HEAD/src/ros_people_model && git fetch --all --tags --prune
git checkout tags/v0.2.0

Checkout the ghost branch of chatbot, hr_launchpad and configs:

cd ${HR_WORKSPACE}/HEAD/src/chatbot && git checkout ghost
cd ${HR_WORKSPACE}/configs && git checkout ghost
cd ${HR_WORKSPACE}/hr_launchpad && git checkout ghost

Clone realsense:

cd ${HR_WORKSPACE}/HEAD/src && git clone https://github.com/intel-ros/realsense.git
cd ${HR_WORKSPACE}/HEAD/src/realsense && git fetch --all --tags --prune
git checkout tags/2.0.3

Build the head stack

hr build head

5. Setup OpenCog

Install octool, OpenCog dependencies and Relex dependencies:

sudo curl -L http://raw.github.com/opencog/ocpkg/master/ocpkg -o /usr/local/bin/octool
sudo chmod +x /usr/local/bin/octool
octool -d
octool -l java

Install OpenCog repos:

hr update opencog

Install Relex Ubuntu dependencies:

cd ${HR_WORKSPACE}/OpenCog/relex/install-scripts && ./install-ubuntu-dependencies.sh

Remove ros-behavior-scripting and checkout the ghost-lai branch of opencog and atomspace:

sudo rm -r ${HR_WORKSPACE}/OpenCog/ros-behavior-scripting
cd ${HR_WORKSPACE}/OpenCog/atomspace && git checkout ghost-lai
cd ${HR_WORKSPACE}/OpenCog/opencog && git checkout ghost-lai

Build OpenCog:

hr clean opencog
hr build opencog

Running

Run the robot first:

hr run --nogui --dev sophia10

Then run ghost_bridge with the avatar:

rosrun ghost_bridge run.sh

Then run ghost_bridge with the actual robot:

rosrun ghost_bridge run.sh robot

For running opencog in gdb use the optional argument in-gdb:

rosrun ghost_bridge run.sh in-gdb

For starting ecan in opencog use the optional argment enable-ecan:

rosrun ghost_bridge run.sh enable-ecan

To stop the robot:

hr stop

To stop ghost_bridge:

rosrun ghost_bridge stop.sh

Tips for running on the Zotac:

PCIe Bus Error

Run the command dmesg, if you get the error PCIe Bus Error: severity=Corrected, type=Physical Layer, id=00e6(Receiver ID), then you can fix it by doing the following.

Run sudo nano /etc/default/grub

And change this line: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

To: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pcie_aspm=off"

Then run sudo update-grub

Design Goals

TODO

Architecture

TODO

Available emotions, gestures and soma states

Emotions:

worry
happySurprise
happy.001
recoil
happyDisgust
happy
surprisedSad
surprised
sad
irritated
happy.002
fearSuprise
fear
engaged
disgustSurprise
disgust.Sad
disgust
contempt
confused
comprehending
bored
awe
angrySad
angryDisgust
angry
amused
none

Gestures:

amused
blink
blink-micro
blink-relaxedblink-sleepy
CryBrows
look-L
look-L.001
look-L.002
look-L.UP
look-R
look-R.001
look-R.002
look-R.UP
lookLR
lookRL
nod-1
nod-2
nod-3
nod-4
nod-5
nod-6
Sad
shake-2
shake-3
shake-4
shake-5,
shake-6
sleep-in
sleep-out
think-browsDown
think-browsDown.001
think-browsDown.002,
think-browsDown.003
think-browsUp
think-browsUp.001
think-browsUp.002
think-browsUp.003,
think-DWN
think-L
think-L.001
think-L.DWN
think-L.UP
think-R
think-R.001,
think-R.DWN
think-R.UP
think-UP
thoughtful
thoughtful.001
WinkR
yawn-1

Soma states:

normal
breathing
normal-saccades
sleep