iSpy_python

I worked on this project while at UNT working in the HiLT Lab as part of the I Spy Project. I graduated in 2015 with my BS and am now at Texas A&M working on my PhD in computer science, so I will no longer be contributing to this code. This code will be maintained by Jacob during his time in the HiLT Lab.

This code simulates a game of I Spy between two players, who in this case are both simulated. We currently play using a set of 17 objects and hopefully we will be able to add new objects to the playspace soon. Imagine that player 1 selects the object to be guessed, and player 2 tries to guess the object. Player 2 learns about objects based on collected descriptions, player 1's answers to questions, and features extracted from images of the objects. As the system simulates games, player 2 should get better at asking questions to determine which object player 1 has selected. This will simultaneously increase accuracy and decrease the number of questions asked to identify the object.

A (very) rough description of how the simulation works can be found here. Another description is here if you feel like reading a conference paper. The version from the paper is not automated like the version in this repository and the formulas are different, but the concept and goals are similar.

If you intend to play the game with a NAO robot, you must install some custom modules. You can find the modules and the installation process in the repo: https://github.com/jacobbrunson/ispy_robot

Setup:

1. Create a database 'iSpy_features' and upload the dump.sql into it
2. Make sure the settings in the config section of main.py match your database credentials, or use the command line flags -u and -p for your username and password and change the config.py file that's created later. Also check the address for the robot, if you're using one.
3. Create a folder within the repo named 'SVM_model_777'. This is where the image models reside.
4. Make sure you use the -s flag to run setup the first time you run the program. If you're planning on using the image models at any point, either use -i now to set them up or run both -s and -i at a later point. The image models won't work without having prior run setup with them turned on.

Run:

• Execute 'python main.py' from within the ispy_python directory
• Use -h if you want to see all the options for running the program

Dependencies:

• Python Setup Tools
• MySQL server
• mysql-python or pymysql
• opencv
• sklearn
• matplotlib
• numpy
• naoqi (if using the robot)
• speech_recognition (if using the robot)
• SoX (command line tool, needed if using the robot)