Skip to content

Demo dataset v2: realistically overgrown inbox (84,000 messages, 1,419 senders) - #18

Merged
zbuc merged 1 commit into
mainfrom
issue-17
Jul 22, 2026
Merged

Demo dataset v2: realistically overgrown inbox (84,000 messages, 1,419 senders)#18
zbuc merged 1 commit into
mainfrom
issue-17

Conversation

@zbuc

@zbuc zbuc commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Closes #17

Implements the dataset v2 design from issue #15.

Summary

  • demo/summary.json regenerated: 84,000 messages across 1,419 senders in 12 categories — social-notification floods at the top (chatterloop 6,842; fotofling 4,517), heavy retail promo and newsletter mass, transactional receipts/finance/platform mail, a 90-sender gray-mail band, 120 human correspondents, and a 400-sender one-off signup tail.
  • demo/dataset_spec.json — the machine-readable spec (category budgets, Zipf exponents, clamps, pinned top senders, brand vocabularies, fixed seed).
  • demo/generate_summary.py — deterministic generator: pinned heroes take exact counts, Zipf tails are jittered/clamped and reconciled to exact per-category budgets, one-off groups use small count mixtures. It asserts every guarantee before writing: totals and sender counts match the spec exactly, addresses are unique, every domain is RFC 2606 reserved.
  • Distribution realism: single top sender ~8% of volume, median sender has 11 messages, 368 senders appear exactly once.

Testing

  • Generator runs clean with all assertions passing; regenerating from the committed spec is byte-identical (fixed seed).
  • Rendered the page in headless Chrome: all 1,419 rows populate, tiles show 84,000 / 1,419 / notifications@chatterloop.example.
  • summary.json is ~103 KB — fine for Pages.

No UI changes; the existing demo banner and constraints (fictional brands, reserved domains, no real data) are unchanged.

Implements the dataset v2 spec from issue #15: 84,000 messages across
1,419 senders in 12 categories -- social notification floods, retail
promos, newsletters, receipts, finance, platform mail, alerts, travel,
gray mail, a small set of human correspondents, and a 400-sender
one-off signup tail.

The dataset is produced deterministically (fixed seed) by the committed
generate_summary.py from dataset_spec.json: pinned hero senders take
exact counts; category tails are Zipf-distributed with clamps and
reconciled to exact per-category budgets; one-off groups use small
count mixtures. The generator validates every spec guarantee before
writing: exact totals and sender counts, address uniqueness, and RFC
2606 reserved domains throughout.

Closes #17
@zbuc
zbuc merged commit 57136ce into main Jul 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Demo dataset v2: realistically overgrown account (84,000 messages)

1 participant