Skip to content

Commit

Permalink
storage: terminate nodes in minority on stats mismatch
Browse files Browse the repository at this point in the history
Following up on cockroachdb#41893, the consistency checker now instructs the
replicas corresponding to the SHA with the lowest multiplicity to
terminate.

Note that this change is a candidate for backporting into 19.2. An "old"
leaseholder will never populate the newly introduced field, and will
always terminate itself. A new leaseholder will populate the field, but
only new nodes will interpret it. In the worst case, the result will be
a consistency failure that does not lead to a fatal error (but collects
RocksDB checkpoints just the same). Since no significant time is usually
spent in a mixed-patch-releases, this is not a concern that outweighs
the benefit of terminating (heuristically) the "right" nodes.

Release note (general improvement): when the replicas within a range
have found to been corrupted, the outliers will be terminated.
Previously, the leaseholder replica would terminate, regardless of
which replicas disagreed with each other. This is expected to curb
the spreading of corrupted data better than the previous approach.
  • Loading branch information
tbg committed Nov 4, 2019
1 parent 30e5bf8 commit 4cf1ee0
Show file tree
Hide file tree
Showing 12 changed files with 1,121 additions and 731 deletions.
96 changes: 86 additions & 10 deletions c-deps/libroach/protos/roachpb/api.pb.cc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

76 changes: 76 additions & 0 deletions c-deps/libroach/protos/roachpb/api.pb.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 4cf1ee0

Please sign in to comment.