Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Better SQL in Jupyter. 📊

Apache Spark 3 - Structured Streaming Course Material

This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Data Engineering examples covering Airflow and Mage for workflows; dbt for BigQuery, Redshift, ClickHouse; Spark and Kafka for Batch/Streaming Processing

Sparglim✨ makes PySpark App Configurable and Deploy Spark Connect Server Easier!

Demonstration of using Apache Spark to build robust ETL pipelines while taking advantage of open source, general purpose cluster computing.

A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.

This is a data processing pipeline that implements an End-to-End Real-Time Geospatial Analytics and Visualization multi-component full-stack solution, using Apache Spark Structured Streaming, Apache Kafka, MongoDB Change Streams, Node.js, React, Uber's Deck.gl and React-Vis, and using the Massachusetts Bay Transportation Authority's (MBTA) APIs …

A structured streaming was applied to the robot data from ROS-Gazebo simulation environment using Apache Spark. Data is collected in Kafka, analyzed by Apache Spark and stored in Cassandra.

An naive anomaly detection and data visualization tool for F1 on board telemetry data.

Sentiment Analysis and Data Visualization

ORM for Apache Spark and DataFrames schema manager

Spark DataFrame with metadata

Liga: Let Data Dance with ML Models

A package that lets you run PySpark SQL on your Protobuf data

This project introduces PySpark, a powerful open-source framework for distributed data processing. We explore its architecture, components, and applications for real-time data analysis.

What people are tweeting about now, in your desired location? Live Streaming of Twitter Data to Spark and Tweet Analysis application on various trends Eg: Trending HashTags, Trending Mentions etc. Location based features supported.