Facebook social data modeling with Scala, HBase, and HPaste
Scala Shell
Latest commit 468e308 Dec 7, 2015 @eugeneiiim eugeneiiim Add tech talk slides
Permalink
Failed to load latest commit information.
bin Scripts to run mapreduce jobs. May 18, 2012
project unused imports May 18, 2012
src personhbasecollection cleanup May 18, 2012
.gitignore initial commit May 17, 2012
README.md add link to slides May 26, 2012
blend-techtalk-may-2012.pdf Add tech talk slides Dec 7, 2015

README.md

social-modeling-starter

Facebook social data modeling with Scala, HBase, and HPaste.

Includes accessing the Facebook Graph API in Scala via Dispatch and Lift JSON, and using HPaste to access HBase and define MapReduce jobs.

Slides at http://blendlabsinc.com/blend-labs-tech-talk-stanford-may-2012.pdf.

Setup

Visit https://developers.facebook.com/tools/explorer?method=GET&path=me . Click "Get Access Token". Select the following permissions:

  • User Data Permissions
    • user_groups
    • user_likes
  • Friends Data Permissions
    • friends_groups
    • friends_likes

Click "Get Access Token", then "Allow". Copy and paste the Access Token. In your console, run:

export FACEBOOK_ACCESS_TOKEN=<your access token>

Download Cloudera's HBase distribution (CDH4 Beta 2) at http://archive.cloudera.com/cdh4/cdh/4/hbase-0.92.1-cdh4.0.0b2.tar.gz.

Unarchive the file and run

bin/start-hbase.sh
sleep 5
bin/hbase shell
hbase> create 'person', 'info', 'like'

To load your Facebook data, run

./bin/load.sh

Two MapReduce calculations are included: "friend likes" and "top likes". To execute these, run

./bin/topLikes.sh
./bin/friendLikes.sh