Popular repositories

1. aws-airline-ingestion-pipeline aws-airline-ingestion-pipeline Public

   Efficiently ingest daily airline data into AWS using a seamless end-to-end pipeline, integrating S3 uploads, Glue schema discovery, Redshift data transformation, and SNS notifications.

   Python 2

2. kafka-mongo-logistics-integration kafka-mongo-logistics-integration Public

   A Python application integrating Kafka and MongoDB for efficient logistics data processing, with Avro serialization, Docker scaling, and an API for seamless interaction.

   Python 1

3. stock-market-kafka-pipeline stock-market-kafka-pipeline Public

   Stock Market Kafka Project: A robust data pipeline leveraging Python, Confluent Kafka, AWS S3, IAM, Glue, and Athena to efficiently process and analyze stock market data. Streamline your workflow f…

   Python 1

4. Order-Tracking-Incremental-Load-Project Order-Tracking-Incremental-Load-Project Public

   The Order Tracking Incremental Load Project automates the integration of order tracking data using Apache Spark in Databricks. Leveraging Google Cloud Storage (GCS) for input, it features efficient…

   Python 1

5. ecommerce-integration-pipeline ecommerce-integration-pipeline Public

   🌐 Seamless E-commerce Data Integration Pipeline using Python, GCP Pub/Sub, DataStax Cassandra, and Pandas. This repository includes scripts to load, publish, and consume data, along with instructio…

   Python 1

6. ETL-pipeline-bank-transaction ETL-pipeline-bank-transaction Public

   Transform daily bank transactions effortlessly with this AWS ETL pipeline. Ingest CSVs to S3, trigger Glue jobs with Lambda, store securely in Parquet, and analyze seamlessly using Athena

   Python 1