Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming pipeline
-
Updated
Jun 21, 2024 - Python
Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming pipeline
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
The project demonstrates an end-to-end data pipeline using Apache Kafka to fetch and stream data from a website. The project is containerized with Docker for streamlined deployment and dependency management.
End-to-end data engineering pipeline with various technologies to ingest real time data.
End-to-end data engineering pipeline with various technologies to ingest real time data.
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Real time streaming of a time series with corresponding forecasts.
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Thesis project: topic categorization and sentiment analysis on twitter with Apache Spark
Estudo sobre o Apache Zookeeper em Python, utilizando a biblioteca Kazoo.
Apache Zookeeper Metric Collector
It contains the manifest to demo the watch feature of the zookeeper.
Ansible playbook to deploy a Zookeeper cluster on Linux Vagrant instance.
Add a description, image, and links to the apache-zookeeper topic page so that developers can more easily learn about it.
To associate your repository with the apache-zookeeper topic, visit your repo's landing page and select "manage topics."