storage/reports: change node liveness considerations #43825
storage/reports: change node liveness considerations
Before this patch, whether a node was "alive" or "dead" did not matter for the under-replicated ranges count in system.replication_stats. This patch makes replicas on dead stores be ignored when counting replicas for purposes of the underreplicated counter. Liveness used to matter for the unavailable count and for the system.critical_localities report (and continues to matter). Note that this change interestingly means that a range can be considered both under-replicated and over-replicated at the same time - if there's too many replicas, but sufficiently many of them are dead. This patch also changes the liveness criteria across the board for the reports that cared about liveness (unavailable ranges, under-replicated ranges and critical localities). The code was buggy in that a decomissioning node was considered live for 5 minutes after it stopped heartbeating its liveness record, whereas a non-decomissioning one was only considered live for a few seconds. The patch fixes it by using the same logic to make liveness determinations as the underreplicated metric does - you're dead as soon as the liveness record expires regardless of decomissioning status. Release note (sql change): Ranges are now considered under-replicated by the system.replication_stats report when one of the replicas is unresponsive (or the respective node is not running).
- Loading branch information
ProTip! Use n and p to navigate between commits in a pull request.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments.