Dimensional and Data Vault Modeling

Dimensional and Data Vault Modeling

Great Generator can generate warehouse-friendly models from domain packs. This helps users teach, demo, and test analytics modeling patterns without building source systems by hand.

Dimensional model generation

Use generate_dimensional_model(...) when you want facts and dimensions.

from great_generator import generate_dimensional_model

model = generate_dimensional_model("ecommerce", scale="small")

customers = model["dim_customer"]
products = model["dim_product"]
sales = model["fact_sales"]

Ecommerce output includes:

• dim_customer
• dim_product
• dim_date
• fact_sales
• fact_payments
• _model_metadata

Banking output includes:

• dim_customer
• dim_account
• dim_merchant
• dim_date
• fact_transactions
• fact_fraud
• _model_metadata

Spark output

model = generate_dimensional_model("banking", engine="spark", scale="medium")

model["fact_transactions"].write.mode("overwrite").parquet(
    "s3://my-bucket/demo/fact_transactions"
)

If Great Generator cannot detect an active Spark session, pass spark=spark explicitly.

Data Vault model generation

Use generate_data_vault_model(...) when you want hubs, links, and satellites.

from great_generator import generate_data_vault_model

vault = generate_data_vault_model("ecommerce", scale="small")

hub_customer = vault["hub_customer"]
hub_order = vault["hub_order"]
link_order_customer = vault["link_order_customer"]
sat_customer_details = vault["sat_customer_details"]

The Data Vault generator derives:

• one hub per keyed domain table
• one satellite per keyed domain table
• one link per foreign-key relationship
• stable hash keys
• load_date
• record_source
• _model_metadata

Why output is returned instead of directly written

The modeling APIs return a dictionary of DataFrames. That gives users full control over where the output goes.

model = generate_dimensional_model("banking", scale="small")
model["fact_transactions"].to_parquet("fact_transactions.parquet")

For Spark:

model = generate_data_vault_model("banking", engine="spark", scale="medium")
model["hub_customer"].write.format("delta").mode("overwrite").save(path)

This lets users write to CSV, JSON, Parquet, Delta, database tables, catalog tables, local paths, S3, ADLS, GCS, DBFS, or any destination their runtime supports.