kafka-streams-app

That's a preview feature, so use it at your own risk.

Prerequisites

Build the trunk of Apache Kafka (0.10) or use the one by Confluent

Create input & output Kafka topics

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic streams-hashtag-input

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic streams-hashtag-count-output

Execute the tweet producer

Set up your Twitter credentials at TweetProducer and execute it to store tweets at streams-hashtag-input queue:

private static final String CONSUMER_KEY = "";
private static final String CONSUMER_SECRET = "";
private static final String TOKEN = "";
private static final String SECRET = "";

Execute the HashtagJob

Read the topic with hashtags and counts

bin/kafka-console-consumer.sh --zookeeper localhost:2181 \
          --topic streams-hashtag-count-output \
          --from-beginning \
          --formatter kafka.tools.DefaultMessageFormatter \
          --property print.key=true \
          --property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer \
          --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer