WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Data processing ang ingestion backend for ViyaDB based on Spark streaming

Basic framework utilities to quickly start writing production ready Apache Spark applications

This project contains the learning and experiments with the Apache Spark.

A simulation to automatically collect weather data and visualize it on maps. Tech stack: Kafka, Spark Streaming, Cassandra, Tableau

A big data project for predicting prices of Uber/Lyft rides depending on the weather

The U.S. Department of Transportation's (DOT) Bureau of Transportation Statistics tracks the on-time performance of domestic flights operated by large air carriers. Summary information on the number of on-time, delayed, canceled, and diverted flights is published in DOT's monthly Air Travel Consumer Report and in this dataset of 2015 flight dela…

Big Data Processing Framework - Unified Data API or SQL on Any Storage

The official repository for the Rock the JVM Spark Streaming course

Maven project cover scala language: sparkml, spark_streaming, spark_dataframe, ... + java language: threadpool, kafka, jpa, timer, request api

Wikipedia updates streaming, transformation and visualisation

Spark Structured Streaming with State Store

Projeto utilizado para aprendizado de Spark Streaming + Scala

Implement some example in scala.

A cutting-edge big data initiative aimed at creating a real-time data pipeline to analyze the popularity and sentiments of trending topics on Twitter.

Kafka API with Java & Sparkstreaming with Scala experiments.

源码剖析Spark2.4

Kinesis Connector for Structured Streaming