Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Code for CHI2014 paper and JSM 2014 presentation
Python TeX Shell
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
paper
twcounty
.gitignore
README.md
logging.conf
requirements.txt
run.sh
setup.py

README.md

twcounty

Code to recreate the results for the paper: Estimating County Health Statistics with Twitter, Aron Culotta, CHI 2014.

Note that this is not intended as a library for Twitter analysis; instead, its main goal is to allow other researchers to reproduce results. This is unfortunately not that easy to do, since it relies on Twitter data (which is hard to share) and LIWC and PERMA lexicons (which are not freely available).

I've also included an iPython notebook for the reweighting experiments presented at JSM 2014.

Installation

  • Dependencies are listed in requirements.txt.
  • Additionally, json2tsv.py uses the tokenizer from QUAC, which as of this writing does not allow easy_install. So, you'll have to install QUAC manually following these instructions, then add quac to your PYTHONPATH.
  • Once all dependencies are installed, run python setup.py develop, which will install some command-line aliases used by run.sh (twcounty-expt, twcounty-json2tsv, twcounty-tsv2feats, twcounty-tsv2stats).

Data

  • Paths to LIWC and PERMA lexicons should be in environmental variables $LIWC and $PERMA. (Unfortunately, these are not free for download.) The expected format is:
%
1     Pronoun
2     Adjective
...
%
he    1
pretty     2
...
  • The code expects tweets in /data/twcounty/json/. Each file should be named after the FIPS of the county from which the tweets originated. The contents should be the raw json returned by the Twitter API. As sharing raw tweets violates Twitter's TOS, please send email to Aron Culotta (aronwc at gmail.com), and I'll send you the list of tweet IDs. You will then have to submit queries to the Twitter API to retrieve the raw content.

Reproducing results

Once everything is installed and the data is in place, you should be able to run the script run.sh. Results will be written to paper/tables and paper/figs. Pickled data files and raw results will be in /data/twcounty/features.

Something went wrong with that request. Please try again.