A reinforcement learning agent that maximizes crowdsourcing quality by teaching and testing.
 Jonathan Bragg, Mausam, and Daniel S. Weld. 2016. Optimal Testing for Crowd Workers. In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS '16). Singapore.
 Jonathan Bragg, Mausam, and Daniel S. Weld. 2015. Learning on the Job: Optimal Instruction for Crowdsourcing. In ICML '15 Workshop on Crowdsourcing and Machine Learning. Lille, France.
After you clone the project, be sure to initialize and update all the submodules. This can be done with
git submodule update --init --recursive.
We recommend installing your environment using Miniconda. Once Miniconda is installed, you can a create virtual environment with the correct dependencies by running
conda env create -f environment.yml python=2.7 -n $ENV
$ENV is the name of your virtual environment.
In order to use policies that utilize POMDPs, you must install a supported POMDP solver. The recommended solver is ZMDP. Once you have installed and built this solver, be sure that you can run
pomdpsol-zmdp from the shell by either aliasing or adding the ZMDP binary to your
$PATH environment variable. One way to do this is
ln -s ~/.bin/pomdpsol-zmdp $ZMDP_BIN_DIR/zmdp
where the ZMDP binary directory is
$ZMDP_BIN_DIR. Then in your
~/.bashrc file, add the line
If you want to use the main experiment or visualization code, you will need to set up a MongoDB database. The application loads the configuration settings for the database from environment variables. An example:
The main end-to-end experiment code lives in
exp.py. Since the project directory is a package, the module should be run from the parent directory of the project.
If you have cloned the repository into a directory named
guru, you can run the main module with
cd .. && python -m guru.exp.
If you would like to use the agent in your own application, you may want to follow the usage in
exp.py as a guideline for how to initialize the agent from
policy.py. More detailed instructions to come.
TODO: Describe visualization application.
Use the provided
Please create an issue, pull request, or send email to jbragg [at] cs.washington.edu.