HDDS-2980. Delete replayed entry from OpenKeyTable during commit #625
Conversation
hanishakoneru
force-pushed
the
HDDS-2980
branch
from
7a91ea3
to
7298d34
Compare
| omClientResponse = new OMKeyCommitResponse(omResponse.build(), | ||
| omKeyInfo, commitKeyRequest.getClientID()); | ||
| omKeyInfo, dbOpenKey); |
There was a problem hiding this comment.
We can still pass here clientID and generate openKey. Like how we used to generate open key before.
There was a problem hiding this comment.
We are anyway computing the openKey using the clientID here. So instead of passing the clientID and computing that again in OMKeyCommitResponse, we can directly pass the openKey.
There was a problem hiding this comment.
I see that we are doing similar thing for ozone key, so that is why my question. Then we can do similar thing for ozone key also then?
There was a problem hiding this comment.
We could. But since we have to pass the omKeyInfo irrespective of whether we pass ozoneKey or not, I didnt change that.
I am ok to have it either way. Will post an update.
hanishakoneru
force-pushed
the
HDDS-2980
branch
from
fb7b289
to
3947e0a
Compare
|
Integration test failure (timeout) is unrelated and passes locally. Re-triggering the checks. |
| @@ -147,11 +146,6 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager, | |||
| throw new OMException("Failed to commit Multipart Upload key, as " + | |||
| openKey + "entry is not found in the openKey table", | |||
| KEY_NOT_FOUND); | |||
| } else { | |||
| // Check the OpenKeyTable if this transaction is a replay of ratis logs. | |||
There was a problem hiding this comment.
Not understood why this check is removed.
There was a problem hiding this comment.
This check was redundant. Irrespective of whether KeyCreate Request was replayed or not, if key+clientID exits in the openKey table, then the CommitPart request should also be executed (same as we do for KeyCommit Request).
If the same Key part was created again, the clientID would be different. Hence the openKey would also be different.
|
Thanks for the review @bharatviswa504 . |
What changes were proposed in this pull request?
During KeyCreate (and S3InitiateMultipartUpload), we do not check the OpenKeyTable if the key already exists. If it does exist and the transaction is replayed, we just override the key in OpenKeyTable. This is done to avoid extra DB reads.
During KeyCommit (or S3MultipartUploadCommit), if the key was already committed, then we do not replay the transaction. This would result in the OpenKeyTable entry to remain in the DB till it is garbage collected.
To avoid storing stale entries in OpenKeyTable, during commit replays, we should check the openKeyTable and delete the entry if it exists.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2980
How was this patch tested?
Added unit test.