Awesome Scalability, Availability, and Stability Back-end Design Patterns

A curated list of selected readings to illustrate Scalability, Availability, and Stability Design Patterns in Back-end Development.

What if your backend went slow?

Understand your problems: performance problem (slow for a single user) or scalability problem (fast for a single user but slow under heavy load) by reviewing some basic design concepts.

What if your backend went down?

"Even if you lose all one day, you can build all over again if you retain your calm!" - Thuan Pham, CTO at Uber Technologies Inc.

Contributing

Please take a look at the contribution guidelines first. Contributions are always welcome!

Contents

• Basic
• Scalability
• Availability
• Stability
• Other Design Aspects
• Awesome Books
• Awesome Talks

Basic

• CAP theorem and the trade-offs
• Scale up vs Scale out
• How to deal with latency
  • Dropbox case: Striving for maximal throughput with acceptable latency
• What is ACID?
• Architecture issues: Bottlenecks, Database, CPU, IO
  • How eBay's Shopping Cart used compression techniques to solve network I/O bottlenecks
• Performance and Scalability patterns
• Advantages and drawbacks of Microservices
• Avoid Overengineering
• Don't Repeat Yourself (DRY)
  • DRY in Django
• Design for Loose-coupling
• Design for Resiliency
• Design for Self-healing when failures occur
• Design for Scale out
  • Design for Scale: Three best practices
• Design for Evolution

Scalability

• Distributed Caching
  • Write-behind and Write-through
  • Eviction Policies
  • Peer-To-Peer Caching
  • Distributed Caching at Netflix with EVCache
  • Robust Memcache Traffic Analyzer at Box.com
  • How Etsy caches: Consistent Hashing and Cache Smearing
• Distributed Logging & Tracing
  • Building DistributedLog at Twitter: High-performance replicated log service
  • Distributed tracing at Pinterest with Pintrace
  • Scalable and reliable log ingestion at Pinterest
  • CERN Accelerator Logging Service with Spark
• Distributed Messaging
  • Understanding When to use RabbitMQ or Apache Kafka
  • Delaying Asynchronous Message Processing with RabbitMQ at Indeed
  • Yelp's Real-time Data Pipeline with Kafka
  • Real-time Deduping at Scale with Kafka-based Pipleline at Tapjoy
• Storage
  • In-memory Storage
    • Optimizing Memcached Efficiency at Quora
    • Real-Time Data Warehouse with MemSQL on Cisco UCS
    • Moving to MemSQL for solving problems at Tapjoy: horizontally scalable, ACID compliant, MySQL compatibility
  • Durable Storage (S3)
    • Reasons for Choosing S3 over HDFS at Databricks
    • S3 in the Data Infrastructure at Airbnb
    • Quantcast File System on Amazon S3
    • Using S3 in Netflix Chukwa
• NoSQL
  • Key-Value Databases (DynamoDB, Voldemort, Manhattan)
    • Scaling Mapbox infrastructure with DynamoDB Streams
    • Manhattan: Twitter’s distributed key-value database
  • Document Databases (Cassandra, Vertica, Sybase IQ)
    • Consistent Hashing in Cassandra
    • When NOT to use Cassandra?
    • Storing Images in Cassandra at Walmart Scale
    • Cassandra at Instagram
    • How Yelp Scaled Ad Analytics with Cassandra
    • How Discord Stores Billions of Messages with Cassandra
  • Graph Databases (MongoDB, CouchDB, Neo4j)
    • eBay: Building Mission-Critical Multi-Data Center Applications with MongoDB
    • MongoDB at Baidu: Multi-Tenant Cluster Storing 200+ Billion Documents across 160 Shards
  • Datastructure Databases (Redis, Hazelcast)
    • How Twitter Uses Redis To Scale
    • How Twitter Uses Redis To Scale - Video
    • Redis in Slack job queue
    • Moving persistent data out of Redis at Github
• RDBMS
  • Why SQL is beating NoSQL, and what this means for the future of data
  • Sharding MySQL at Pinterest
  • How Airbnb Partitioned Main MySQL Database in Two Weeks
  • Replication is the Key for Scalability & High Availability
  • How Twitch uses PostgreSQL
  • Scaling MySQL-based financial reporting system at Airbnb
  • Scaling to 100M at Wix: MySQL is a Better NoSQL
  • Why Uber Engineering Switched from Postgres to MySQL
  • Handling Growth with Postgres at Instagram
• HTTP Caching
  • Reverse Proxy (Nginx, Varnish, Squid, rack-cache)
  • CDN (Akamai, Amazon CloudFront)
    • NASA - Streaming 4K Live from the International Space Station Using CloudFront
• Concurrency
  • Message-Passing Concurrency
  • Software Transactional Memory
  • Dataflow Concurrency
  • Shared-State Concurrency
• Event-Driven Architecture
  • Messaging
    • Publish-Subscribe
      • Autoscaling Pub/Sub Consumers at Spotify
    • Point-to-Point
    • Store-Forward
    • Request-Reply
  • Actors: Fire-forget and Fire-Receive-Eventually
  • Enterprise Service Bus
  • Domain Events
  • Event Stream Processing
  • Event Sourcing
  • Command & Query Responsibility Segregation (CQRS)
• Load Balancing
  • Round-robin Allocation
  • Random Allocation
  • Weighted Allocation
  • Dynamic Load Balancing
    • Work Stealing
  • Consistent Hashing
  • UDP Load Balancing
  • Cloud Load Balancing
    • AWS ELB: Application Load Balancer, Network Load Balancer, Classic Load Balancer
      • AWS ELB issues at Asana, 2012
    • Google Cloud Load Balancing
• Parallel Computing
  • SPMD (Single Program Multiple Data): The Genetic Pattern
  • Master/Worker Pattern
  • Loop Parallelism Pattern: Extracting parallel tasks from loops
  • Fork/Join Pattern: Good for recursive data processing
  • MapReduce Pattern: Born for Big Data
  • Parallelize the rendering of web pages: Use case of Yelp.com

Availability

• Fail-over
• Replication
  • Master-Slave
  • Tree Replication
  • Master-Master
  • Buddy Replication

Stability

• Circuit Breaker
• Always use timeouts (if possible)
• Let it crash/Supervisors: Embrace failure as a natural state in the life-cycle of the application
• Crash early: An error now is better than a response tomorrow
• Bulkheads: Partition and tolerate failure in one part
• Steady state: Always put logs on separate disk
• Throttling: Maintain a steady pace
• Multi-clustering: Improving Resiliency and Stability of a Large-scale Monolithic API Service at LinkedIn

Others

• Scalable Gaming Patterns on AWS (Sep 2017)
• Building a Modern Bank Backend
• Best Practices For Horizontal Application Scaling (by Shekhar Gulati - OpenShift)

Books

• The Art of Scalability
• Designing Data-Intensive Applications
• Web Scalability for Startup Engineers

Talks

• Harvard CS75 - Lecture 9: Scalability
• How We've Scaled Dropbox - Kevin Modzelewski, Back-end Engineer at Dropbox
• Lessons of Scale at Facebook - Bobby Johnson, Director of Engineering at Facebook
• Scaling Instagram Infrastructure - Lisa Guo, Instagram Engineering
• Scaling Pinterest - Marty Weiner, Pinterest’s founding engineer
• Designing for Failure: Scaling Uber's Backend by Breaking Everything - Matt Ranney, Chief Systems Architect at Uber

Special Thanks

• Jonas Bonér, CTO at Lightbend, for the original inspiration