Pinned Loading

1. sparkify-etl-airflow-docker sparkify-etl-airflow-docker Public

   End-to-end data engineering project: ETL pipeline for Sparkify using Apache Airflow, Docker, Terraform, and AWS Redshift Serverless. Automates infrastructure, orchestration, and data loading from S…

   Python

2. sparkify-redshift-etl-pipeline sparkify-redshift-etl-pipeline Public

   This project builds an end-to-end cloud data warehouse on AWS Redshift for Sparkify, a fictional music app. It extracts raw song and log data from Amazon S3, loads it into staging tables, and trans…

   Python

3. stedi-datalake-full-terraform stedi-datalake-full-terraform Public

   A fully Terraform-managed AWS data lake pipeline for IoT sensor data analytics. Converts raw landing data into trusted and curated layers for machine learning using Glue, S3, and Athena. Originally…

   Python

4. stedi-datalake-glue-visual-etl stedi-datalake-glue-visual-etl Public

   End-to-end data lakehouse pipeline on AWS using Glue, S3, Athena, and Terraform to curate human balance data for machine learning.

   Python

5. Sparkify-Cassandra Sparkify-Cassandra Public

   Designed a NoSQL data model using Apache Cassandra for Sparkify, a music streaming startup. Cleaned CSV logs with Python, modeled denormalized tables based on analytical queries, and built ETL scri…

   Python

6. Bank_Scoop Bank_Scoop Public

   BankScoop is a Python-based ETL pipeline that scrapes global bank rankings, converts market caps to multiple currencies, stores the results in a SQLite database, logs every step, and generates a cl…

   Python