This is a self-study project. My goals for this project are
- learn to write better networking code
- learn to write better Scala code
- develop a detailed understanding of RAFT
To get started, use
$ sbt compile
I use the Cake Pattern throughout the project for dependency injection. The assembly module lives in
src/scala/com/jimjh/raft/package.scala, which might be a good place to start exploring the code.
$ sbt scroogeGen
Here's a simplified description of the various classes and traits I intend to have. In practice, there will be a proliferation of traits and classes to enable dependency injection. Refer to the ScalaDoc for a detailed and up-to-date description of each package, class, and trait.
Users of this library are expected to
- provide an application that implements the
- provide a logger that implements the slf4j api, and
- launch instances of
ScalaDocs may be generated using
$ sbt doc
The package overview might be a good place to start.
The client makes RPCs to the server to execute some command on the state machine. Details TBD.
Each server will be implemented using a state machine that governs the transitions between the follower, leader, and candidate states. Servers have the following attributes:
- votedFor (persistent) - do we need one for each term?
- state - one of
Its implementation shall be divided into a
ClientService (handles RPCs from clients) and a
RPCs between RAFT nodes.)
The log is responsible for
- file I/O,
- compaction (snapshotting),
- recovery etc.
lastApplied:long(volatile/persistent, depends on snapshot),
Commands are applied by forwarding them to the delegate.
Application defines an application interface. Applications implement this interface to receive committed commands from
On success, the
lastApplied counter is incremented. If the application raises an exception, the server will be
def apply(cmd: String, args: Array[String])
As documented in the RAFT paper, it's up to the application to prevent duplicate executions of the same command e.g. assign unique serial numbers to each command and ignore commands that have been executed.
Triggers election timeouts on the server.
Triggers periodic heartbeats on the server.
An effort will be made to use dependency injection across the library. This will allow tests to provide mocks that can be used to simulate various network failures.
To run the tests, use
$ sbt test
- Phase 1: Skeleton classes, RPCs
- Phase 2: Timeouts, Leadership Election
- Phase 3: Log Replication and Application
- Phase 3.5: Persistent State
- Phase 4: Membership Changes
- Phase 5: Log Compaction (Snapshotting)
- Onagro, D., Ousterhout, J. In Search of an Understandable Consensus Algorithm, Stanford University, 2014