HDDS-9432. Perform individual delete key instead of delete range function#5454
HDDS-9432. Perform individual delete key instead of delete range function#5454swamirishi merged 10 commits intoapache:masterfrom
Conversation
hemantk-12
left a comment
hemantk-12
left a comment
There was a problem hiding this comment.
Thanks for the patch @swamirishi
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/ThrowableFunction.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
| } | ||
|
|
||
| // Clean up deletedDirectoryTable | ||
| deleteRangeInclusive(omMetadataManager.getDeletedDirTable(), |
There was a problem hiding this comment.
Delete this method. It is not used anymore.
There was a problem hiding this comment.
Let us keep this method. We might have to dig further as to why deleteRange is failing.
There was a problem hiding this comment.
Please add a comment to not remove it. Someone might just go over and remove it since it is not used anywhere.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hemantk-12
left a comment
hemantk-12
left a comment
There was a problem hiding this comment.
Overall looks good to me.
Left couple of nit comments.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Outdated
Show resolved
Hide resolved
prashantpogde
left a comment
prashantpogde
left a comment
There was a problem hiding this comment.
thanks @swamirishi for this PR. Overall the changes look good to me.
Lets test/verify if this avoids the rocksDB crash that we are seeing in deletRange. If not, we are probably better off with the use of deleteRange API.
Sure |
|
Thank you @hemantk-12 @prashantpogde for reviewing the patch. |
…tion (apache#5454) * HDDS-9432. Perform individual delete key instead of delete range function * HDDS-9432. Fix issue * HDDS-9432. Fix checkstyle issue * HDDS-9432. Address review comments * HDDS-9432. Add test cases * HDDS-9432. Fix test case * HDDS-9432. Address review comment * HDDS-9432. Address review comment * HDDS-9432. Fix checkstyle * HDDS-9432. Fix javadocs
…e range function (apache#5454) (cherry picked from commit 33140cc) Change-Id: I9e3f209bb0b8f5cbede4ee83012b600683a15dde
3 participants
What changes were proposed in this pull request?
Currently rocksDb sst file becomes corrupt in case of delete range function while creating snapshot. Speculating the corruption could be because of delete range function. When a snapshot for a bucket is created, all keys corresponding to the bucket in the deletedTable & deletedDirTable are removed from active object store after creating the rocksdb checkpoint.
What is the link to the Apache JIRA:
https://issues.apache.org/jira/browse/HDDS-9432
How was this patch tested?
Existing unit tests. Issue is not reproducible