data-engineering-projects

Data Warehouse Modelling

1. Sales Warehouse: Fundamental implementation of a Star Schema for sales data.

   • Focus: Star Schema, Surrogate Keys, B-Tree Indexes, Python + SQLite.

2. Subscription Warehouse: Tracking user history with SCD Type 2 and Snowflake Schema.

   • Focus: SCD (Slowly Changing Dimensions) Type 2, Snowflake Schema, DuckDB (Columnar Storage).

3. Order Fulfillment Warehouse: Tracking e-commerce orders through a multi-stage pipeline.

   • Focus: Accumulating Snapshot Fact Table, Role-Playing Dimensions, Degenerate Dimensions, Junk Dimensions, DuckDB.

4. Inventory Snapshot Warehouse: Monitoring stock levels and category hierarchies over time.

   • Focus: Periodic Snapshot Fact Table, Parent-Child Hierarchies, Recursive CTEs, Semi-Additive Measures.

5. Healthcare Bridge Warehouse: Handling Many-to-Many relationships and semantic abstractions.

   • Focus: Bridging Tables, Multi-valued Dimensions, Semantic Layer (Views), Revenue Allocation.

Streaming pipeline

1. Climate Stream Analytics

• Focus: Cloud Run, Pub/Sub, Dataflow, BigQuery, Cloud Scheduler , Artifact Registry