Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Online social media research and computational journalism project by the Journalism and Media Studies Centre at the University of Hong Kong. Led by Dr. King-wa Fu (Principal Investigator). Developer until August 2012: Cedric Sam
Python Shell
branch: master
Failed to load latest commit information.
googleplus Google+ support for comments
qq sql dumps, qq and google+ activities
sinafetch support for OAuth 2. user timeline + status only
sinaweibo sql dumps, qq and google+ activities
AUTHORS Google+, README, AUTHORS
README Google+, README, AUTHORS
blogs.parse.py Periodic update
facebook.graph.py Commit Facebook
facebook.search.py Commit Facebook
facebook.token.README.txt Readme for generating the Facebook token
facebook_pull.sql Commit Facebook
hkforums.search.py Periodic update
mypass.py commit
sinagetter.sh Superseded by sinaweibo.oauth.py
sinamostretweeted.sh occasional commit
sinamostretweeted_firstpass.py occasional commit
sinamostretweeted_secondpass.py occasional commit
sinapeople.sh MAXTRIES=10
sinareposts.sh support for listid-based reports
sinastorage.py Not used anymore. Superseded by sinaweibo.oauth.py
sinatrace.py occasional commit
sinaurl.archiver.py occasional commit
sinaurl.py monthly update
sinaweibo.deleted.py occasional commit
sinaweibo.lucene.py Periodic update
sinaweibo.oauth.py occasional commit
sinaweibo.search.py Periodic update
sinaweibo.sql first commit
sinaweibo2.oauth.py added support for getting reposts in orig posts
sinaweibo_pull.sql sinaweibo sql schema
social.lucene.py Periodic update
twitter.db.py occasional commit
twitter.geograb.py first commit
twitter.oauth.py occasional commit
twitter.rel.py Twitter relations
twitter.users.py keys
twitter_pull.sql don't know...
watch.facebook.check.sh occasional commit

README

=====================================================================================================================
Code repository for the computational journalism project (Social) of the Journalism and Media Studies Centre at HKU
---------------------------------------------------------------------------------------------------------------------

We developed tools to pull data from Sina Weibo, Twitter and Facebook, as well as blogs and HK forums.

Since the second part of 2011, the Sina Weibo tools are Python scripts that leverage the librairies provided by Sina (weibopy) to access their API through OAuth2.

twitter.oauth.py and facebook.oauth.py perform various pull operations from Twitter and Facebook. The OAuth token and secret, Facebook ID, should be in your version of mypass.py. Usage information is generally available when running the script. facebook.search.py is used for searching (a way of discovering new contents by keyword search).

twitter_pull.sql and facebook_pull.sql should create the postgresql databases needed to store the data of these aforementioned scripts.

In late November 2011, we added scripts to fetch data from the Google+ and the QQ Weibo API. The database schema are available in their respective sub-directories.

These scripts are developed under Linux (Ubuntu Lucid 10.04). I used Python 2.6.5 and my version of Postgresql is 8.4 (with Postgis 1.5).
Something went wrong with that request. Please try again.