This project is a data engineering and visualization solution that collects, processes, and visualizes weather data using Docker, MongoDB, Apache Airflow, and Power BI.
The project aims to create an end-to-end data pipeline that fetches weather data from the OpenWeather API, processes it using Python, stores the results in MongoDB, and visualizes the data using Power BI. The pipeline is orchestrated using Apache Airflow, and all components are containerized using Docker.
- Data collection from OpenWeather API
- Data processing and transformation using simple python code
- Data storage in MongoDB
- Data pipeline orchestration with Apache Airflow
- Data visualization using Power BI
- Docker: Containerization and deployment
- MongoDB: NoSQL database for storing weather data
- Apache Airflow: Orchestration of data pipeline tasks
- Power BI: Data visualization and reporting
