Test scenario implementation roadmap

[jopemachine]

One of the goals of this crate is to provide a robust, high-level Raft implementation that is ready to use.
To achieve this goal, we need to write more tests for various scenarios.

Integration tests

Should pass

• Cluster bootstrap testing
  • Boostrapping based on node's static rule
  • WAL bootstrap
  • WAL snapshot bootstrap
• Vote and Leader election
• Request rerouting
• Log replication
• Log compaction
• Simple membership change
• Membership change based on Joint consensus

Should fail

Writing a list of failing test cases would be useful because it helps identify which failures are expected behavior and which are bugs.

• Timeout on put requests when a quorum is broken.
• Timeout on new node join attempts when a quorum is broken.
• Handling put requests during leader failover.
• Timeout on join requests during leader failover.
• Failures occurring while creating a snapshot.
• Whether a new proposal is rejected when there is a pending configuration change, and if it is processed normally after the pending configuration change is resolved.
• If a node not in the peers sends a message to the cluster, it should raise a PeerNotFound error.
• A configuration change request containing only an AddNode entry for itself times out.

Unit tests

Types that require unit tests

• RaftNode
• HeedStorage (from raft-rs)

Others

• Doc testing
• Client API testing
• Create a simple shell script (test-cluster.sh) that verifies if the cluster is functioning correctly for each example.