Online social media research and computational journalism project by the Journalism and Media Studies Centre at the University of Hong Kong. Led by Dr. King-wa Fu (Principal Investigator). Developer until August 2012: Cedric Sam
Python Shell
Failed to load latest commit information.
googleplus Google+ support for comments Mar 6, 2012
qq sql dumps, qq and google+ activities Dec 2, 2011
sinafetch support for OAuth 2. user timeline + status only Jan 29, 2013
sinaweibo sql dumps, qq and google+ activities Dec 2, 2011
AUTHORS Google+, README, AUTHORS Dec 8, 2011
README Google+, README, AUTHORS Dec 8, 2011
blogs.parse.py Periodic update Oct 18, 2011
facebook.graph.py Commit Facebook Jul 30, 2012
facebook.search.py Commit Facebook Jul 30, 2012
facebook.token.README.txt Readme for generating the Facebook token May 16, 2012
facebook_pull.sql Commit Facebook Jul 30, 2012
hkforums.search.py Periodic update Oct 18, 2011
mypass.py commit Mar 1, 2011
sinagetter.sh Superseded by sinaweibo.oauth.py May 24, 2011
sinamostretweeted.sh occasional commit Nov 16, 2011
sinamostretweeted_firstpass.py occasional commit Nov 16, 2011
sinamostretweeted_secondpass.py occasional commit Nov 16, 2011
sinapeople.sh MAXTRIES=10 Mar 1, 2011
sinareposts.sh support for listid-based reports Mar 1, 2011
sinastorage.py Not used anymore. Superseded by sinaweibo.oauth.py May 24, 2011
sinatrace.py occasional commit Nov 16, 2011
sinaurl.archiver.py occasional commit Jul 30, 2012
sinaurl.py monthly update Apr 11, 2011
sinaweibo.deleted.py occasional commit Nov 16, 2011
sinaweibo.lucene.py Periodic update Oct 18, 2011
sinaweibo.oauth.py occasional commit Jul 30, 2012
sinaweibo.search.py Periodic update Oct 18, 2011
sinaweibo.sql first commit Oct 25, 2010
sinaweibo2.oauth.py added support for getting reposts in orig posts Jan 11, 2013
sinaweibo_pull.sql sinaweibo sql schema Jul 30, 2012
social.lucene.py Periodic update Oct 18, 2011
twitter.db.py occasional commit Jul 26, 2011
twitter.geograb.py first commit Oct 25, 2010
twitter.oauth.py occasional commit Nov 16, 2011
twitter.rel.py Twitter relations Jul 30, 2012
twitter.users.py keys Dec 21, 2010
twitter_pull.sql don't know... Dec 6, 2010
watch.facebook.check.sh occasional commit Jul 26, 2011

README

=====================================================================================================================
Code repository for the computational journalism project (Social) of the Journalism and Media Studies Centre at HKU
---------------------------------------------------------------------------------------------------------------------

We developed tools to pull data from Sina Weibo, Twitter and Facebook, as well as blogs and HK forums.

Since the second part of 2011, the Sina Weibo tools are Python scripts that leverage the librairies provided by Sina (weibopy) to access their API through OAuth2.

twitter.oauth.py and facebook.oauth.py perform various pull operations from Twitter and Facebook. The OAuth token and secret, Facebook ID, should be in your version of mypass.py. Usage information is generally available when running the script. facebook.search.py is used for searching (a way of discovering new contents by keyword search).

twitter_pull.sql and facebook_pull.sql should create the postgresql databases needed to store the data of these aforementioned scripts.

In late November 2011, we added scripts to fetch data from the Google+ and the QQ Weibo API. The database schema are available in their respective sub-directories.

These scripts are developed under Linux (Ubuntu Lucid 10.04). I used Python 2.6.5 and my version of Postgresql is 8.4 (with Postgis 1.5).