No description, website, or topics provided.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app
scripts
storm_test
.gitignore
LICENSE
README.md
ebay_bot.py
ebay_save.py

README.md

This document is a work in progress

Introduction to YeezyScore

Description: This project compares Storm Trident with Spark Streaming. While the two come from differing ends of the processing software universe, these two features in particular are considered 'microbatching' stream processing technologies.


Project for Data Engineering Fellowship at Insight Data Science '15C


Table of Contents:

  1. Introduction
  2. Technologies
  3. Application

Introduction

Initially I wanted to work on a sneaker analytics related project, but then knowing nothing about realtime processing software and hearing vague advice of, "pick the right tool for the job" led me on a path of comparing available options. I started with the two most popular choices: Apache Storm and Apache Spark. Further investigation into each led me to find out that both have micro-batching capabilities. This was the basis of comparison.


Technologies

Application

Here's an example of using stream processing software to create a trendwatching dashboard

Preview

Data comes from eBay API.