distributed_systems_readings

a list of papers, conference, books, mooc, Q&A and other stuffs for distributed systems

issues for more materials are welcome.

---

papers

theories related to distributed systems, maily basic concepts/consensus algorithms/formal methods etc.

• basic concepts/introductions

  • Distributed Systems for Fun and Profit (strongly recommend)
  • Notes on distributed systems for young bloods
  • A Note on Distributed Systems
  • Time, clocks, and the ordering of events in a distributed system
  • Fundamentals of distributed computing: A practical tour of vector clock systems
  • HLC: Hybrid Logical Clocks
  • Virtual Time and Global States of Distributed Systems
  • Timestamps in Message-Passing Systems That Preserve the Partial Ordering
  • Distributed snapshots: determining global states of distributed systems
  • Development of the domain name system
  • Rediscovering-Distributed-System

• consistency/fault-tolerence/replication

  • consistency

    • The part-time parliament
    • The Byzantine Generals Problem
    • Paxos Made Simple
    • Viewstamped replication: A new primary copy method to support highly-available distributed systems
    • The Chubby lock service for loosely-coupled distributed systems
    • Paxos Made Live: An Engineering Perspective
    • Paxos for System Builders
    • ZooKeeper: Wait-free Coordination for Internet-scale Systems
    • Zab : High-performance broadcast for primary-backup systems
    • ZooKeeper ’ s atomic broadcast protocol : Theory and practice
    • In Search of an Understandable Consensus Algorithm
    • PAXOS Made Transparent
    • Revisiting the PAXOS algorithm
    • The Paxos Family of Consensus Protocols
    • Multi-Paxos: An Implementation and Evaluation
    • Consensus on transaction commit
    • Consistency in Distributed Storage Systems An Overview of Models, Metrics and Measurement Approaches
    • Base: An Acid Alternative
    • Eventually Consistent
    • Consensus in the Cloud: Paxos Systems Demystified

  • fault-tolerence/replication

    • Impossibility of Distributed Consensus With One Faulty Process
    • Implementing fault-tolerant services using the state machine approach: a tutorial
    • Remus: High Availability via Asynchronous Virtual Machine Replication
    • Perspectives on the CAP Theorem
    • Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services
    • CAP Twelve Years Later

  • leader election

    • Leader election
    • Bully algorithms
    • Leader Election Algorithms

• formal methods

  • The temporal logic of actions
  • A TLA+ proof system
  • The Model Checker SPIN
  • Use of Formal Methods at Amazon Web Services
  • MODIST: Transparent Model Checking of Unmodified Distributed Systems
  • 模型检测在分布式系统中的应用

• others

  • Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web
  • Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications
  • Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems

distributed storage

• fs

  • Design and implementation of the Sun network filesystem
  • The Google file system
  • The Hadoop distributed file system
  • Ceph : A Scalable , High-Performance Distributed File System
  • Finding a needle in Haystack: Facebook's photo storage

• database

  • Bigtable
  • Hadoop-HBase for large-scale data
  • Dynamo
  • Spanner: Google’s Globally-Distributed Database

distributed computing

• cluster management

  • Large-scale cluster management at Google with Borg
  • Omega: flexible, scalable schedulers for large compute clusters
  • Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center
  • Yarn

• computing

  • Dryad : Distributed Data-Parallel Programs from Sequential Building Blocks
  • MapReduce : Simplified Data Processing on Large Clusters
  • Pregel: a system for large-scale graph processing
  • Dremel: Interactive Analysis of Web-Scale Datasets
  • Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing
  • Storm@twitter
  • GraphX: Graph Processing in a Distributed Dataflow Framework
  • Introducing Apache Giraph for Large Scale Graph Processing
  • Large-Scale Distributed Graph Computing Systems : An Experimental Evaluation

parallelism and concurrency

• Models for Parallel Computing : Review and Perspectives
• Actors: A Model of Concurrent Computation in Distributed Systems
• Communicating sequential processes
• Parallel Algorithms Lecture Notes
• DTHREADS: Efficient and Deterministic Multithreading
• Kendo: efficient deterministic multithreading in software

others

• Kafka : a Distributed Messaging System for Log Processing

---

conference

• List of distributed computing conferences

---

books

• Replication: Theory and Practice
• Distributed Systems: Concepts and Design
• Distributed Systems: Principles and Paradigms
• Distributed Systems: An Algorithmic Approach
• Distributed Algorithms: An Intuitive Approach
• Distributed Computing: Principles, Algorithms, and Systems

---

mooc

• MIT 6.824: Distributed Systems
• CMU 15-440: Distributed Systems Syllabus
• MIT 6.852/18.437 Distributed Algorithms
• MIT 6.S897: Large-Scale Systems
• CS 525 Spring 2015 Advanced Distributed Systems
• CS–745/845: Formal Specification and Verification of Systems

---

some nice blog posts

• UNDERSTANDING PAXOS
• The Log: What every software engineer should know about real-time data's unifying abstraction
• Consensus Protocols: Two-Phase Commit
• Consensus Protocols: Three-phase Commit
• Three-Phase Commit Protocol
• Consensus Protocols: A Paxos Implementation
• Consensus Protocols: Paxos
• FLP and CAP are not the same
• Consistency and availability in Amazon's Dynamo
• Distributed systems theory for the distributed systems engineer
• PAXOS/MULTI-PAXOS ALGORITHM
• EVENTUAL CONSISTENCY
• The Essential Leslie Lamport
• The Essential Nancy Lynch
• The Essential Barbara Liskov
• Viewstamped Replication Revisited

---

Q&A

• What are the seminal papers in distributed systems
• 分布式系统领域有哪些经典论文