System Design Method

System Design Method

The lens every topic playbook applies. Read before any single design.

System design is a chain of decisions under constraint. You decide what to ingest, what to store, what to compute, what to serve. Each stage kills bad cost and keeps useful signal. The best design is not the one that does the most. It is the one that chooses best what to do.

The shape of any system

Almost every nontrivial system has four capabilities plus two support planes.

Capability	Search engine example	General
Discover / ingest	crawler	get data in (API, events, uploads, crawl)
Understand / model	parser, extractor	parse, validate, enrich, normalize
Organize	inverted index	store for efficient query (index, schema, partition)
Serve	query path	answer requests under a latency budget

Support planes, almost always present:

• Metadata / policy — rules, scheduling, dedup, config, quotas.
• Observability / control — metrics, debugging, replay, backfill, allow/blocklists.

The three pillars (Kleppmann, DDIA)

Score every design against these. They are the non-functional spine.

Reliability

Works correctly under fault: hardware failure, software bug, human error. Design for failure — Werner Vogels: "everything fails all the time". The tools are stability patterns (Nygard): timeouts, retries with exponential backoff, circuit breakers, bulkheads, idempotency. A reliable system assumes its dependencies will fail and degrades on purpose rather than by accident.

Scalability

Handles growth in load. The discipline: define load parameters first (QPS, payload size, fan-out, read/write ratio), then describe performance under that load (p50/p95/p99, throughput). Prefer horizontal over vertical scaling. Partition by a key that avoids hot spots. "Scalable" is not a property of a system; it is the answer to "if load grows by X, what is our plan?"

Maintainability

Operable, simple, evolvable. John Ousterhout's A Philosophy of Software Design: build deep modules — simple interfaces hiding significant implementation. Complexity is the enemy; it accumulates in dependencies and obscurity. Observability is part of the design, not an afterthought.

Numbers every engineer should know (Jeff Dean)

Use for back-of-envelope. Order of magnitude, not exact.

Operation	Time
L1 cache reference	~1 ns
Branch mispredict	~5 ns
Main memory reference	~100 ns
Compress 1 KB	~2 us
SSD random read	~16 us
Read 1 MB sequentially from memory	~10 us
Read 1 MB sequentially from SSD	~50 us
Round trip within same datacenter	~0.5 ms
Read 1 MB sequentially from disk	~5 ms
Disk seek	~2 ms
Round trip CA to Netherlands	~150 ms

Sizing math, always shown:

QPS         = DAU x actions/day / 86400
peak QPS    = avg x (2 to 10)
storage     = records x bytes/record x replication x retention
bandwidth   = QPS x payload

If you cannot do the math, you do not understand the scale yet.

The 13-stage method

An SDD walks these. The template mirrors them.

1. Problem + context — one-line frame. Who, scale, internal vs web-scale.
2. Requirements — functional, plus non-functional with numbers (latency SLO, throughput, availability, consistency model, cost ceiling).
3. Mental model — end-to-end flow, numbered or mermaid. See the whole path before any component.
4. High-level architecture — components + mermaid + the two support planes.
5. Deep dives — the 2-3 components that carry the risk: data structures, algorithm, hard trade-off. This is where staff separates from senior.
6. Data + storage — separate store by function and access pattern. KV/wide-column for random updates, object storage for blobs, search index for text, relational for transactions. Favor immutability (Helland: events over mutable state).
7. Scale + partitioning — sharding strategy, replication, rebalancing. Owner-per-partition for coordination. Stateless workers autoscale; stateful needs leadership + replicas.
8. Consistency + failure — pick a consistency model honestly (eventual is fine if it converges). Enumerate failure modes and how the design resists each. "What breaks first?"
9. Observability + ops — metrics per stage; the debugging tools that must exist (inspect one record's lifecycle, explain one request's result).
10. Security + compliance — least data stored, sandbox untrusted input, sanitize parsing, respect external policy.
11. Incremental plan — vertical slice first (prove end-to-end on small scope, reuse proven engines), then efficiency/quality, then real scale, then advanced.
12. Trade-offs — coverage vs quality, freshness vs cost, recall vs latency, complexity vs delivery speed, centralize vs partition.
13. Open questions / design review — what a reviewer should interrogate.

Trade-off discipline

Never present one option as obvious. Name the alternative, the axis, the choice:

Chose X over Y because {axis} matters more here, given {constraint}.

A design with no stated trade-off is a design that hid one.

Build pragmatically

Do not reinvent the hard parts if your differentiation lives elsewhere. Use a proven engine (Lucene, Postgres, Kafka, a managed queue) for the treacherous-detail layer; spend the quarters on the pipeline and logic that is actually your edge. Reinvent only the part that is the product.

Consistency, time, and ordering

Leslie Lamport's Time, Clocks, and the Ordering of Events in a Distributed System (1978) is the root of distributed reasoning: in a distributed system there is no single global "now"; you reason about causal ordering, not wall-clock ordering. This underlies eventual consistency, vector clocks, and why "just use timestamps" is a trap (clock skew). Helland's Life Beyond Distributed Transactions extends it to the practical conclusion: at scale you give up cross-entity ACID and design around independent, idempotent, eventually-reconciled units.

The CAP and PACELC framing

Under a network partition (P) you choose availability (A) or consistency (C) — CAP (Brewer). PACELC adds: else (E), when there is no partition, you trade latency (L) against consistency (C). Most real designs are "AP under partition, and trade latency for consistency in normal operation". Say which corner you sit in and why; it is rarely all-or-nothing per system — it is per operation.

The canon

Cite when it sharpens a point.

Person	Idea	Source
Martin Kleppmann	reliability/scalability/maintainability; load params before perf	DDIA (2017)
Jeff Dean, Sanjay Ghemawat	numbers everyone knows; MapReduce-shaped batch	LADIS 2009; OSDI 2004
Werner Vogels	design for failure; eventual consistency at scale	Dynamo (SOSP 2007)
Pat Helland	immutability; life beyond distributed transactions	CIDR 2007; 2015
Michael Nygard	circuit breaker, bulkhead, timeout, backoff	Release It! (2018)
John Ousterhout	deep modules, simple interfaces	PoSD (2018)
Leslie Lamport	causal ordering, no global clock	CACM 1978
Eric Brewer	CAP theorem	PODC 2000
Sam Newman	service boundaries along business capability	Building Microservices
Gregor Hohpe	the architect elevator: connect engine-room trade-off to business stake	2020

The harness connection

This agent is itself a harness (Böckeler/Fowler). Guides = feedforward, sensors + evals = feedback, humans on the loop. The same control split the design needs: computational controls (tests, linters, schema checks) and inferential controls (semantic review). Both, always.

References

• Martin Kleppmann, Designing Data-Intensive Applications, O'Reilly, 2017.
• Jeff Dean, Designs, Lessons and Advice from Building Large Distributed Systems, LADIS, 2009.
• Dean & Ghemawat, MapReduce: Simplified Data Processing on Large Clusters, OSDI, 2004.
• DeCandia et al., Dynamo: Amazon's Highly Available Key-value Store, SOSP, 2007.
• Pat Helland, Life Beyond Distributed Transactions, CIDR, 2007.
• Pat Helland, Immutability Changes Everything, ACM Queue, 2015.
• Michael Nygard, Release It!, 2nd ed., Pragmatic Bookshelf, 2018.
• John Ousterhout, A Philosophy of Software Design, 2018.
• Leslie Lamport, Time, Clocks, and the Ordering of Events in a Distributed System, CACM, 1978.
• Eric Brewer, Towards Robust Distributed Systems (CAP), PODC keynote, 2000.
• Birgitta Böckeler, Harness engineering for coding agent users, martinfowler.com, 2026.