Airflow Deployment on AWS ECS Fargate Using Cloudformation
-
Updated
Jun 22, 2022 - HTML
Airflow Deployment on AWS ECS Fargate Using Cloudformation
Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.
Apply Data Engineering to Personal Finance
This ETL (Extract, Transform, Load) project employs several Python libraries, including Airflow, Soda, Polars, YData Profiling, DuckDB, Requests, Loguru, and Google Cloud to streamline the extraction, transformation, and loading of CSV datasets from the U.S. government's data repository at https://catalog.data.gov.
Live COVID-19 tracker with Airflow
A data pipeline to extract News articles from BBC News, storing it to MongoDB, performing NLP analysis (Topic modeling, sentiment analysis), and orchestrating with airflow.
As a Data Engineer for a fictional E-commerce startup, this project addresses the task of analyzing the web server logs to find the number of product pages visited and the number of items in the cart.
I generated the list of tables using Schemaspy and hosting it on my github. Definitely helps in understanding the backend tables of airflow.
White and Red Wine classification using logistic regression
Ask Ubuntu Logs analysis with PySpark on GCP | Pipeline with Airflow (Cloud Composer)
Siby Abin - Senior Data Engineer
End-to-end ML pipeline model and drift monitoring for Loan Eligibility classification.
A maquina preditiva, analisa a possibilidade de um colaborador vier deixar a empresa, e quais são os fatores principais de sua saida
Photovoltaic System Modeling and Predictions
Add a description, image, and links to the airflow topic page so that developers can more easily learn about it.
To associate your repository with the airflow topic, visit your repo's landing page and select "manage topics."