Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
distributed twitter search engine
branch: ui

This branch is even with javasoze:ui

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
build
config
lib
project
src
README.md
sbt

README.md

What is Chirper?

Chirper is a real time tweet search engine (written in Scala) using open source technology built by the LinkedIn SNA team:

Although the amount of code is minimal, the system can scale horizontally infinitely by leveraging the distributed system above.

Build and run the system:

From the top level directory, e.g. ~/chirper

  1. sbt update (do this once to setup the probject)
  2. Make sure your twitter username/password is set in the config file
  3. sbt run

You will see a list of classes that ca be run:

Multiple main classes detected, select one to run:

[1] com.linkedin.chirper.streamer.ChirpStream
[2] com.linkedin.chirper.services.ZookeeperRunner
[3] com.linkedin.chirper.services.KafkaRunner
[4] com.linkedin.chirper.services.VoldemortRunner
[5] com.linkedin.chirper.search.ChirpSearchNode

Enter number:

Select a class to run (you'll need to run one per console window)

The order of class to be run are:

  1. [2] - Zookeeper
  2. [3] - Kafka
  3. [4] - Voldemort
  4. [1] - Streamer (Make sure your twitter username/password is set in the config file)
  5. [5] - Search Node

Each of the components are pre-configured, details see configs.

The last thing to run is a Restful servlet and interacts with the system (Comes with a beautiful UI):

chirper$ sbt
[info] Building project Chirper 1.0 against Scala 2.8.0
[info]    using ChirperStreamerProject with sbt 0.7.4 and Scala 2.7.7
> jetty-restart

Now you can point to:

Something went wrong with that request. Please try again.