-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: quorum requirement for DeleteMarkers and parity upgraded objects #14248
Conversation
DeleteMarkers do not have a default quorum, i.e it is possible that DeleteMarkers were created with n/2+1 quorum as well to make sure that we satisfy situations such as those we need to make sure delete markers only expect n/2 read quorum. Additionally we should also look at additional metadata on the actual objects that might have been "erasure" upgraded with new parity when disks are down. In such a scenario do not default to the standard storage class parity, instead use the parityBlocks present on the FileInfo to ensure that we are dealing with the correct quorum for READs and DELETEs.
Mint Automation
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍🏼 In general relying on default parity will lead to bugs when reading (since it obviously is variable). Writing of course knows which quorum to expect.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from minio#14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
…s() (#14273) Deleting bulk objects had an issue since the relevant versionID is not passed through the layers to ensure that the dangling object purge actually works cleanly. This is a continuation of quorum related error returned by multi-object delete API from #14248 This PR ensures that we pass down correct information as well as extend the scope of dangling object detection.
Description
fix: quorum requirement for DeleteMarkers and parity upgraded objects
Motivation and Context
DeleteMarkers do not have a default quorum, i.e it is possible that
DeleteMarkers were created with n/2+1 quorum as well to make sure
that we satisfy situations such as those we need to make sure delete
markers only expect n/2 read quorum.
Additionally, we should also look at additional metadata on the
actual objects that might have been "erasure" upgraded with new
parity when disks are down.
In such a scenario do not default to the standard storage class
parity, instead use the parityBlocks present on the FileInfo to
ensure that we are dealing with the correct quorum for READs and
DELETEs.
How to test this PR?
Few datasets are needed for example
inspect.324c74d9.zip
inspect.384b43df.zip
Without this fix, these objects shall return
503 quorum
error, however with this fix you shallsee they all succeeding.
Types of changes
Checklist:
commit-id
orPR #
here)