Conversational agent which can sense user emotion and remember prior conversations
Switch branches/tags
Nothing to show
Clone or download
Latest commit 068ffe0 Aug 14, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
att_faces Code Upload Aug 14, 2017
corpus Code Upload Aug 14, 2017
current_models Code Upload Aug 14, 2017
haar_cascade Code Upload Aug 14, 2017
.gitignore Code Upload Aug 14, 2017
README.md Code Upload Aug 14, 2017
chatbot.py Code Upload Aug 14, 2017
context_recognition.py Code Upload Aug 14, 2017
emotion_recognition.py Code Upload Aug 14, 2017
face_data.csv Code Upload Aug 14, 2017
face_recognizer.py Code Upload Aug 14, 2017
name_data.csv Code Upload Aug 14, 2017
speech_to_text.py Code Upload Aug 14, 2017
topic_extraction.py Code Upload Aug 14, 2017
user_interface.py Code Upload Aug 14, 2017

README.md

Chat Bot w/ Emotion Recognition and History Recollection

This goal of this project is to create a conversational agent that can sense a users current emotional state, remember prior conversations and respond appropriately to the user given this additional information.

Project was created for a graduate class, Perception in Robotics, at ASU. The project was a joing effort between three contributors: Scott Freitas, Benjamin Clayton, Josh Feinglass

The program has a few hardware requirements that must be met in order for it to run properly:

(1) Webcam (2) Microphone (3) NVIDIA graphics card with ~8GB of RAM

In order to run the program:

(1) Get a Microsoft Bing Voice Recognition API key and enter it into the second line under main in the chatbot.py file.

(2) Run chatbot.py

Note: The code has only been tested under Ubuntu 14.04. In addition, the visual interface will likely not display properly for all monitor resolutions. Only tested under 2560 x 1695.

Youtube Demonstration Video:

https://www.youtube.com/watch?v=Ze4XzxQIzNs

Website references used to help create this project:

Emotion Recognition:

(1) https://github.com/tflearn/tflearn/blob/master/examples/images/residual_network_cifar10.py

Context Recognition:

(2) http://www.cs.duke.edu/courses/spring14/compsci290/assignments/lab02.html

(3) http://stackoverflow.com/questions/12118720/python-tf-idf-cosine-to-find-document-similarity

GUI:

(4)http://pyglet.readthedocs.io/en/pyglet-1.2-maintenance/programming_guide/image.html#displaying-images

(5)http://stackoverflow.com/questions/37354309/pyglet-hello-world-example-doesnt-show-label-until-a-key-is-pressed

Voice-to-text:

(6) https://pypi.python.org/pypi/SpeechRecognition/

Text-to-speech:

(7) http://pythonprogramminglanguage.com/text-to-speech/

Face recognition:

(8) http://docs.opencv.org/2.4/modules/contrib/doc/facerec/tutorial/facerec_video_recognition.html

Text topic analysis:

(9) http://scikit-learn.org/stable/auto_examples/applications/topics_extraction_with_nmf_lda.html

(10) https://papers.nips.cc/paper/3902-online-learning-for-latent-dirichlet-allocation.pdf