Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Contoso

A synthetic retail sales dataset for the fictional Contoso company, as eight Parquet files with a data-dict.yaml describing them.

data-dict.yaml     data dictionary: tables, columns, relationships, glossary
make-parquet.R     script that produces parquet/ from the R package
parquet/           sales, orders, orderrows, product, customer, store, calendar, fx

Start with data-dict.yaml for what the data means — the grain of each table, what every column holds, how the tables join, and the traps. This file covers only where the data came from and how to rebuild it.

Provenance

Converted from the R package contoso version 2.1.0, whose bundled data frames were exported to Parquet essentially verbatim (one exception below). The package data in turn comes from the sql-bi Contoso Data Generator V2, whose documentation explains several features of the data that the R package's help pages don't.

All of it is synthetic. Customer names, addresses, birthdays, occupations, employers, vehicles, and coordinates are generated filler, not real people. Nothing is derived from real trading activity, so observed distributions, seasonality, and margins are artefacts of the generator's configuration and should not be read as realistic retail behaviour.

Size

This is the small snapshot: 7,794 sales lines. The R package can stream larger versions (medium ~2.3M, large ~47M, mega ~237M sales rows) from Cloudflare R2 via create_contoso_duckdb(size = ...). The schemas are the same across sizes, but the row counts, ranges, and examples in data-dict.yaml describe the small snapshot only.

Recreating the Parquet files

Rscript make-parquet.R

Run it from this directory; it writes parquet/, overwriting what's there. Requires R with the contoso (>= 2.1.0) and nanoparquet packages:

install.packages(c("contoso", "nanoparquet"))

The output is byte-identical for a given contoso version, so regenerating on a clean checkout leaves git status empty. To confirm the dictionary still matches the data afterwards:

data-dict validate-spec data-dict.yaml   # structurally valid
data-dict validate-meta data-dict.yaml   # column names and types match
data-dict validate-data data-dict.yaml   # values match declared types

One deliberate departure from the R package

customer is the only table not exported as-is. The package ships it fanned out to sales-line grain: 7,794 rows, one per row of sales, with each customer's row repeated byte-for-byte once per sales line they appear in. That makes customer_key non-unique, so any join to it silently multiplies rows.

It's an upstream packaging artefact rather than meaningful data, so make-parquet.R collapses the duplicates to 3,165 rows, one per customer, restoring customer_key as a genuine primary key. The script asserts that deduplicating leaves exactly one row per customer, so it fails loudly rather than dropping data if a future package version varies an attribute within a customer. No other table is altered.

Where the R package's documentation is wrong

Seven column descriptions in the R package's help pages contradict the data. data-dict.yaml follows the data and flags each discrepancy on the column concerned, but if you're reading ?contoso::calendar alongside it, these are the ones that will bite:

  • calendar.working_day_number is a cumulative count of working days, not a 0/1 flag. It runs 0–1003.
  • calendar.year_quarter_number and year_month_number are continuous period counters (year*4 + quarter, year*12 + month), not 1..4 and not YYYYMM. January 2021 is 24253.
  • calendar.day_of_week_number starts the week on Sunday (1 = Sunday), not Monday.
  • calendar.year_quarter and year_month are formatted Q1-2021 and January 2021, not 2025 Q1 and 2025-03.
  • store.description is always Contoso Store <state>, never a store format like "Flagship" or "Outlet".
  • store.status never takes the value Open; it's null for stores trading normally.
  • product.weight_unit is grams/ounces/pounds, never kg.

What the generator's documentation explains

Several patterns that look arbitrary in the data turn out to be documented parameters of the generator. This is knowledge about how the data was manufactured, so it's recorded here rather than in data-dict.yaml — a dictionary should describe the data as found. But it's what confirmed several readings, and it's worth knowing which patterns carry no meaning:

  • PPC (price percent change) — a per-category price curve over the period. This is the factor relating transacted prices to product.price, and finding it named in the docs is what established that the money columns are USD rather than converted to each order's currency.
  • DiscountWeights — the discount distribution, which is why discounts land on exact whole percentages (0–14%).
  • OnlinePerCent — share of orders sold online. The generator's store -1 is emitted as store_key 999999.
  • DeliveryDateLambdaWeights — online delivery is "one day plus a random number", which is why the online lag is always 1–8 days and never 0.
  • GeoAreas — leaf-level regions with no hierarchy above them, which is why geo_area_key resolves to nothing.
  • Subcatlinks — configured cross-subcategory purchase affinities, so products co-occur on orders by design. Any "market basket" association you find is an artefact of this parameter.
  • AnnualSpikes / OneTimeSpikes — volume multipliers for chosen periods, so order volume is deliberately non-uniform over the year. Don't read it as real seasonality. Note also that the first and last months of the range are partial (orders start 2021-05-18 and stop 2024-04-20), which depresses May and April counts for reasons unrelated to demand.

customer.start_date and end_date are not explained by any of this. The generator documents no such fields, and their meaning remains undetermined.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages