Efficient data transformation and modeling framework that is backwards compatible with dbt.
-
Updated
Jul 10, 2024 - Python
Efficient data transformation and modeling framework that is backwards compatible with dbt.
ETL workloads for excel sheet data-sources
Firestore - DynamoDB migration
end-to-end data engineering project to get insights from PyPi using python, duckdb, MotherDuck & Evidence
Future City Development: Data Management Platform
Data Warehouse IBM 🔡🔢🏭
Code and data for the Modern Polars book
Azure Data Engineering and Machine Learning: Helper Functions and Code
This project is designed to fetch historical stock data for a predefined list of companies and store it in a MySQL database. It utilizes Python, Spark, and the Alpha Vantage API to perform data extraction, transformation, and loading (ETL) tasks.
This course is designed to provide learners with the fundamental skills needed for data engineering using Python. The objective is to introduce anyone interested in the topic to Python's data engineering-related features.
This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.
MIMICAI - Exploring MIMIC Data Using LLM and Open-WebUI
This project involves creating an automated script to extract, transform, and load (ETL) GDP data of all countries.
Stream Processing of website click data using Kafka and monitored and visualised using Prometheus and Grafana
An analysis tool that automates the process of data extraction, cleaning, analysis, and visualization. This tool is built using Python and Streamlit, providing an intuitive web interface for users to upload datasets and receive comprehensive analysis and visualizations.
Deploy Selenium And Merge It With PyWebIo = WebApp For Scraping News Web Count(10) Then Tr Into Ar Lang
Data engineering project solution for order data analytics using python and sqllite
Add a description, image, and links to the dataengineering topic page so that developers can more easily learn about it.
To associate your repository with the dataengineering topic, visit your repo's landing page and select "manage topics."