vopr: state checker knows about all commits in liveness mode #1707
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We currently switch to liveness mode only when everything is committed by the cluster, which is not as live as we ideally want to get, but that's a start. In particular, when liveness mode checks for too many corruptions, it makes use of the fact that for every op the state checker knows the correct header. Before upgrades, this invariant was enforced by checking that every request received a reply. But upgrade requests don't generate replies! So it might be the case that, at the time we transition to liveness mode, there's still an uncommitted op! Fix this by checking that at least one replica upgraded all the way up to the latest release. Seed: 2749715711070932366 Closes: #1697
sentientwaffle
approved these changes
Mar 15, 2024
|
Huh, it seems like a similar failure still triggers: #1712 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We currently switch to liveness mode only when everything is committed by the cluster, which is not as live as we ideally want to get, but that's a start.
In particular, when liveness mode checks for too many corruptions, it makes use of the fact that for every op the state checker knows the correct header.
Before upgrades, this invariant was enforced by checking that every request received a reply. But upgrade requests don't generate replies!
So it might be the case that, at the time we transition to liveness mode, there's still an uncommitted op!
Fix this by checking that at least one replica upgraded all the way up to the latest release.
Seed: 2749715711070932366
Closes: #1697