Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Django framework for crowdsourcing complex tasks using MTurk http://smus.com/crowdforge
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
CrowdForge: Crowdsourcing Complex Work (released for non-commercial use creativecommons.org/licenses/by-nc/3.0/) Getting CrowdForge to run locally: 1. Prerequisites: python, django, boto (tested with python-2.7, django-1.2.5 and boto-1.8d). Known issues with boto-1.9b. - NOTE: There's a bug in boto-1.8d that you need to tweak. Change connection.py:466 to say value = self.connection.get_utf8_value(value) (instead of) value = self.get_utf8_value(value) 2. Download: go to https://github.com/borismus/CrowdForge and get the tarball. 3. Extract: let's say to ~/crowdforge 4. Configure: tweak settings.py - specify AWS keys found at URL: https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key - specify absolute URL to your templates directory 5. Create database: run ./manage.py syncdb, create new account 6. Run server: run ./manage.py runserver 7. Test! 8. Setup cron jobs! Testing CrowdForge 1. Point browser to localhost:8000/admin (or whatever your production server URL is) and login. 2. Add a new Problem instance. Give it a name and these parameters - flow: SimpleFlow, - partition: create article - map: collect a fact - reduce: write a paragraph 3. Run `./manage.py poll` manually, and a HIT should be created (check the MTurk server) - expected output: 4. Do a sample HIT from the new HIT group 5. Run `./manage.py poll` again, and a Result should be created locally (look in db or check http://localhost:8000/turk/problem/YOUR_PROBLEM_ID) - expected output: results retrieved [<Result: Result for "Create an Article Ou... #141KMADQXJEYF9EQWEG9UA7S6J5JEW">] Setting up Cron Jobs 0. Basically, you want to run `./manage.py poll` periodically 1. So just tweak your crontab - `crontab -e` - Create a line that says something like "*/15 * * * * /path/to/manage.py poll >> /path/to/crowdforge.log" Getting CrowdForge deployed on a production server: 1. Get CrowdForge running locally (as per instructions above) 2. Get django and crowdforge running on a public facing server 3. Tweak settings.py: - specify external URL of your server in settings.py - switch from sandbox to production - change the database engine to be mysql or postgresql 4. Resync database 5. Do a test 6. Setup cron jobs Advanced: Make your own Hit Types 1. Open management console (localhost:8000/admin or whatever) 2. Add a new Hit Type. 3. Specify title, description and body. These fields can all be parametrized, depending on your flow. Advanced: Make your own flow 1. Open crowdforge/flows.py 2. Create your own subclass of Flow. See SimpleFlow for an example. 3. Don't forget to register your new flow using register('MyNewFlow', MyNewFlow)