This ETL (Extract, Transform, Load) project employs several Python libraries, including Airflow, Soda, Polars, YData Profiling, DuckDB, Requests, Loguru, and Google Cloud to streamline the extraction, transformation, and loading of CSV datasets from the U.S. government's data repository at https://catalog.data.gov.

Comparison between several Python data profile libraries.

This repository contains interactive data dashboards generated for the Karnataka Police Datathon 2024. The analysis focuses on various aspects of accident data, including details about accused individuals, FIR insights, accident reports, complainant information, and victim insights.

Data profiling y-data profile, Data staging (Staging tables), Talend for ETL jobs, MySQL validations Dimensional model (Target tables), Facts and Dimensions, Mapping document explaining the source column name and where it finally maps to target column, Stage to Target, Document all transformations if any

Read, Visualize, Automate and..