Commits on Mar 18, 2019

1. storage: don't send historical Raft log with snapshots …

   Assume a leaseholder wants to send a (Raft or preemptive) snapshot to a follower.
   Say the leaseholder's log ranges from 100 to 200, and we assume that the size
   (in bytes) of this log is 200mb. All of the log is successfully
   committed and applied, and is thus reflected in the snapshot data.
   
   Prior to this change, we would still send the 200mb of log entries along
   with the snapshot, even though the snapshot itself already reflected
   them.
   
   After this change, we won't send any log entries along with the
   snapshot, as sanity would suggest we would.
   
   We were unable to make this change because up until recently, the Raft
   truncated state (which dictates the first log index) was replicated and
   consistency checked; this was changed in cockroachdb#34660. The migration
   introduced there makes it straightforward to omit a prefix of the
   log in snapshots, as done in this commit.
   
   Somewhere down the road (19.2?) we should localize all the log
   truncation decisions and simplify all this further. I suspect
   that in doing so we can avoid tracking the size of the Raft log
   in the first place; all we really need for this is some mechanism
   that makes sure that an "idle" replica truncates its logs. With
   unreplicated truncation, this becomes cheap enough to "just do".
   
   Release note (bug fix): Remove historical log entries from Raft snapshots.
   These log entries could lead to failed snapshots with a message such as:
   
       snapshot failed: aborting snapshot because raft log is too large
       (25911051 bytes after processing 7 of 37 entries)

   

   tbg committed Mar 18, 2019
   Configuration menu

   • View commit details
   • Copy full SHA for 56b79f1
   • Browse repository at this point

   Copy the full SHA

   56b79f1 View commit details

   Browse the repository at this point in the history