Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Project Karen · GitHub license Python Versions Read the Docs GitHub release (latest by date)

This project is dedicated to building a "Synthetic Human" which is called Karen (for now) for which we have assigned the female gender pronoun of "she". She has visual face recognition (opencv/opencv), speech transcription (mozilla/deepspeech), and speech synthesis (festival). Karen is written in Python and is targeted primarily at the single board computer (SBC) platforms like the Raspberry Pi.

Visit our main site: https://projectkaren.ai/

Read the docs: https://docs.projectkaren.ai/

Karen's Architecture

Karen's architecture is divided into two main components: Containers and Devices. The containers focus on communication between other containers and devices are designed to control input and output operations. The most important container is the Brain which is a special type of container as it collects data and provides the skill engine for reacting to inputs. While a Brain does support all the methods of a normal container it is recommended to create a separate container to store all your devices.

Python Module Overview

Class/Object Description TCP Port
karen.containers.Brain Main service for processing I/O. 8080
karen.containers.DeviceContainer Secondary service for devices. 8081

Python Device Module Overview

Class/Object Description
karen.devices.Speaker Audio output device for text-to-speech conversion
karen.devices.Listener Microphone device for speech-to-text conversion
karen.devices.Watcher Video/Camera device for object recognition
karen.panels.RaspiPanel Panel device designed for Raspberry Pi 7" screen @ 1024x600

In version 0.8.0 and later you are no longer required to install the brain, device, and the built-in plugins separately.

Installation

Karen is available through pip, but to use the built-in devices there are a few extra libraries you may require. Please visit the Basic Install page for more details.

# Install PIP (Python package manager) if not already installed
sudo apt-get -y install python3-pip

# Install the required system packages
sudo apt-get -y install \
  libfann2 \
  python3-fann2 \
  python3-pyaudio \
  python3-pyqt5 \
  python3-dev \
  festival \
  festvox-us-slt-hts  \
  libportaudio2 \
  libasound2-dev \
  libatlas-base-dev \
  cmake

# Optionally create your local environment and then activate it
python3 -m venv /path/to/virtual/env --system-site-packages
/path/to/virtual/env/bin/activate

# Install the required build libraries
python3 -m pip install scikit-build 

# Install required runtime libraries
python3 -m pip install urllib3 \
  requests \
  netifaces \
  numpy \
  deepspeech \
  pyaudio \
  webrtcvad \
  opencv-contrib-python \
  Pillow \
  padatious \
  fann2 \
  traceback

# Install the karen module
python3 -m pip install karen

To start execute as follows:

python3 -m karen

You can disable any of the built-in devices or containers with --disable-builtin-[speaker, watcher, listener, panels, brain, container]. Use the --help option for full listing of command line options including specifying a custom configuration file.

NOTE: The program will create/save a version of the configuration to ~/.karen/config.json along with any other data elements it requires for operation. The configuration file is fairly powerful and will allow you to add/remove devices and containers for custom configurations including 3rd party devices or custom skills.

Troubleshooting: "Cannot find FANN libs"

If you encounter an error trying to install the karen module on the Raspberry Pi then you may need to add a symlink to the library FANN library. This is due to a bug/miss in the "find_fann" function within the Python FANN2 library as it doesn't look for the ARM architecture out-of-the-box. To fix it run the following:

ln -s /usr/lib/arm-linux-gnueabihf/libdoublefann.so.2 /usr/local/bin/libdoublefann.so

Web Control Panel

If everything is working properly you should be able to point your device to the web control panel running on the Brain engine to test it out. The default URL is:

» http://localhost:8080/

Control Panel

Demo running on Raspberry Pi

Project Karen


Help & Support

Help and additional details is available at https://projectkaren.ai