基于spark-streaming的实时数仓

Statistics of coin’s volume in real time on Binance in the last 1 hour. Use Kafka to get data from Binance API, then Spark Streaming reads data stream.

基于Spark的电影推荐系统

My Technical Blog Repo

End-to-end real-time credit card transactions application. Made with Kafka, Spark, Bootstrap, ECharts, RxJS.

This project will show an auto-updated map with the people interaction during COVID19 in the US using big data technologies to analysis a real-time stream of Twitter data.

This project uses machine learning to categorize and prioritize airline user tweets based on content and sentiment. The goal is to reduce airlines' workload and provide personalized, empathetic responses to users. By training a sentiment analysis model, airlines can better understand customers' needs and improve their overall service on Twitter.

A Data pipeline and a analytics dashboard that tracks the number of data updates in interval of 1 mins. The result is shown as a graph on the web browser. The input data is first supplied to Kafka, from which it gets streamlined to Spark streaming where some processing happens. Then the final data is updated on the browser.

Analysis for a streaming daily retail data using Spark structured streaming and querying this data to get insights

Jupyter notebooks for filtering Kafka data with Spark Streaming.

Just a few proyects with py, r, spark, storm, and others...

As a Data Engineer for a fictional E-commerce startup, this project addresses the task of analyzing the web server logs to find the number of product pages visited and the number of items in the cart.

Python projects related to the course- Algorithms for Data Guided Business Intelligence

MBIT Big Data 2019-2020 Apache Spark Case Study (M-02 DC-02)

Twitter Spark Streaming using PySpark

A repository for ipython notebook backup

Structured Spark Streaming workshop for Codess 2018

scripts for starting up a kafka+spark cluster on a slurm manager

Official repo for DK908 - IOT Big Data Processing