No description, website, or topics provided.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

This document is a work in progress

Introduction to YeezyScore

Description: This project compares Storm Trident with Spark Streaming. While the two come from differing ends of the processing software universe, these two features in particular are considered 'microbatching' stream processing technologies.

Project for Data Engineering Fellowship at Insight Data Science '15C

Table of Contents:

  1. Introduction
  2. Technologies
  3. Application


Initially I wanted to work on a sneaker analytics related project, but then knowing nothing about realtime processing software and hearing vague advice of, "pick the right tool for the job" led me on a path of comparing available options. I started with the two most popular choices: Apache Storm and Apache Spark. Further investigation into each led me to find out that both have micro-batching capabilities. This was the basis of comparison.



Here's an example of using stream processing software to create a trendwatching dashboard


Data comes from eBay API.