incrementalloading

Here are 2 public repositories matching this topic...

Sanjayvk98 / Spotify-End-to-End-Azure-Data-Engineering-Pipeline

A metadata-driven, CDC-based Medallion architecture pipeline that ingests a Spotify-style streaming warehouse from Azure SQL, processes it through Bronze → Silver → Gold using Databricks Autoloader and Delta Live Tables, and models it as a Star Schema in Unity Catalog — with SCD Type 2 history, Git-based CI/CD, and automated failure alerting.

python jinja2 azure pyspark azure-functions dlt jinja2-templates azuredatafactory azuredatabricks delta-lake azuresql starschema scd-type-2 azurelogicapp deltalivetables incrementalloading

Updated Jul 25, 2026
Python

Aashish-Pandey07 / etl-framework

Star

Configuration-driven ETL framework in Python — YAML-defined pipelines, plugin architecture, incremental loading, data quality checks, DuckDB warehouse.

python yaml etl pipelines data-engineering warehouse duckdb dataqualitycheck incrementalloading

Updated Aug 2, 2026
Python

Improve this page

Add a description, image, and links to the incrementalloading topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the incrementalloading topic, visit your repo's landing page and select "manage topics."

Learn more