Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
distributed twitter search engine
branch: master

This branch is even with javasoze:master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
build
config
lib
project
src
README.md
sbt

README.md

What is Chirper?

(http://javasoze.github.com/chirper/)

Chirper is a real time tweet search engine (written in Scala) using open source technology built by the LinkedIn SNA team:

Although the amount of code is minimal, the system can scale horizontally infinitely by leveraging the distributed systems above. See this blog post on design considerations.

Build and run the system:

From the top level directory, e.g. ~/chirper

  1. sbt update (do this once to setup the probject)
  2. Make sure your twitter username/password is set in the config file
  3. sbt run

You will see a list of classes that ca be run:

Multiple main classes detected, select one to run:

[1] com.linkedin.chirper.streamer.ChirpStream
[2] com.linkedin.chirper.services.ZookeeperRunner
[3] com.linkedin.chirper.services.KafkaRunner
[4] com.linkedin.chirper.services.VoldemortRunner
[5] com.linkedin.chirper.search.ChirpSearchNode

Enter number:

Select a class to run (you'll need to run one per console window)

The order of class to be run are:

  1. [2] - Zookeeper
  2. [3] - Kafka
  3. [4] - Voldemort
  4. [1] - Streamer
  5. [5] - Search Node

Each of the components are pre-configured, details see configs.

The last thing to run is a Restful servlet that interacts with the system (Comes with a beautiful UI):

chirper$ sbt
[info] Building project Chirper 1.0 against Scala 2.8.0
[info]    using ChirperStreamerProject with sbt 0.7.4 and Scala 2.7.7
> jetty-run

Now you can point to:

Something went wrong with that request. Please try again.