A Dockerized Kafka system for streaming server metrics and load balancer logs, with data processing using Spark and storage in a relational database and Hadoop.

Big data computing tasks conducted with PySpark. The problems involve MapReduce and Streaming algorithms.

This project gets data from Spotify API , ingests into kafka for streaming and processes it through spark streaming. All this is done on Azure.

数仓理论和项目（离线、实时）

a streaming app and a dashboard for visualizing cryptocurrency data fetched from the CoinGecko API. The streaming app retrieves real-time cryptocurrency information using Spark Streaming and stores it in a PostgreSQL database.

A real-time sales data analysis Application using Spark Structured Streaming, Kafka as a messaging system, PostgreSQL as a storage for processed data, and Superset for creating a dashboard.

The project aims to design and implement a real-time movie recommendation system using the EK Stack (Elasticsearch and Kibana), Kafka, and a personalized recommendation API to enhance the user experience on Jay-Zz Entertainment's streaming platform.

Developed a real-time streaming analytics pipeline using Apache Spark to calculate and store KPIs for e-commerce sales data, including total volume of sales, orders per minute, rate of return, and average transaction size. Used Spark Streaming to read data from Kafka, Spark SQL to calculate KPIs, and Spark DataFrame to write KPIs to JSON files.

Blog on github pages with Jekyll

Big Data Spark Hadoop Kafka Flink Spark Streaming

全套大数据基础学习教程，包含最基础的centos、maven。大数据主要包含hdfs、mr、yarn、hbase、kafka、scala、sparkcore、sparkstreaming、sparksql。教程包含所有的源代码演示以及在线文档说明。

Spark 2.x 案例操作：Scala版本与 Java1.8lambda版代码示例。涵盖Spark核心技术操作SparkCore、SparkSql、SparkStreaming。同时提供了Spark高级性能优化、序列化、广播变量、数据倾斜、算子优化、JVM优化、troubleshooting、数据倾斜解决方案。是多年来根据工作积累整理出来！

SparkStreaming新手友好向模板，简化SparkStreaming开发

Spark in Action

Spark in Action, 2e - chapter 10 - Ingestion through structured streaming

Big Data Project - SSML - Spark Streaming for Machine Learning

This repository contains files, codes and markdown documents for "big data from scratch" writings on my blog (z-ing.net)

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

spark salesforce connector

电影推荐系统，包括基于ALS、LFM的离线推荐、实时推荐，基于Spark