🎄 Advent of Data

14 Days of Real-World Data Engineering Challenges with dbt + DuckDB

Learn data engineering by fixing the same problems you'll face in production.

Quick Start

# Setup
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

# Verify
dbt debug

# Start Day 1
cat docs/day01.md
dbt seed --select day01
# Build your solution in models/day01_schema_chaos/
dbt run --select day01
dbt test --select day01

The Challenges

Day	Challenge	Difficulty	Topic
1	Schema Chaos	⭐	JSON parsing, mixed formats
2	Duplicate Detective	⭐	Deduplication, window functions
3	Late Arrivals	⭐⭐	Late-arriving data, date math
4	NULL Nightmare	⭐⭐	NULL handling, data cleaning
5	SCD Surprise	⭐⭐⭐	Slowly changing dimensions
6	Testing Triumph	⭐⭐	dbt tests, data quality
7	Incremental Intro	⭐⭐⭐	Incremental models
8	Fan-out Fiasco	⭐⭐	Join explosions, grain
9	Timezone Terror	⭐⭐⭐	Timezone conversions
10	Source Freshness	⭐⭐	Freshness tests, monitoring
11	Data Contracts	⭐⭐⭐	Schema enforcement
12	Metrics Mayhem	⭐⭐⭐	Metrics layer, semantic models
13	Refactoring Rage	⭐⭐	Safe refactoring, impact analysis
14	The Great Migration	⭐⭐⭐⭐	Full pipeline migration

Project Structure

advent-of-data/
├── docs/           # Challenge descriptions
├── seeds/          # Raw data per day
├── models/         # Your solutions go here
├── solutions/      # Reference solutions
├── tests/          # Custom tests
└── macros/         # Reusable SQL

Why dbt + DuckDB?

• Zero infrastructure — Runs locally, no cloud needed
• Real skills — Same patterns used in production
• Fast — DuckDB is blazingly fast for development

---

🎄 Happy data engineering!