SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.

🏆 Spark4You Design patterns

Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

Cryptocurrency monitoring system. Includes data scraping (Selenium), processing (Spark) and visualization (grafana). Pub-Sub messaging using Kafka, persistence of processed data using PostgreSQL. Deployment with Docker Compose.

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

An open source framework for building data analytic applications.

a suite of benchmark applications for distributed data stream processing systems

Sentiment Analysis on streaming data and batch data from Reddit

Structured data streaming using Spark’ s Structured Streaming API ,kafka for data ingestion and cassandra for data storing

This repo contains Big Data Project, its about "Real Time Twitter Sentiment Analysis via Kafka, Spark Streaming, MongoDB and Django Dashboard".

Real-Time Sentiment Analysis on Twitter Streams is a web application that categorizes tweets into sentiments like Negative, Positive, Neutral, or Irrelevant. Built using Apache Kafka , Spark and PySpark ML models, it offers real-time analysis capabilities.

Schema Registry

Data Streaming with Debezium, Kafka, Spark Streaming, Delta Lake, and MinIO

This is a comprehensive solution for real-time football analytics, leveraging Apache Spark execution on yarn for both streaming and batch processing, Hadoop HDFS for distributed storage, Kafka for real-time data ingestion, rethinkdb for live data updates , a custom built search engine and Next.js for data visualization.

Ophelia a PySpark analytics wrapper.

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs