-
Notifications
You must be signed in to change notification settings - Fork 275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DR] Compute CRC for backup recovery and verification #2788
Conversation
ambry-server/src/main/java/com/github/ambry/server/AmbryRequests.java
Outdated
Show resolved
Hide resolved
ambry-server/src/main/java/com/github/ambry/server/AmbryRequests.java
Outdated
Show resolved
Hide resolved
@@ -241,6 +245,13 @@ private void fetchDataFromReadSet(MessageFormatMetrics metrics) throws MessageFo | |||
} | |||
} | |||
|
|||
public long getBlobCRCOffset(MessageReadSet messages, int idx) throws MessageFormatException, IOException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is a better way to know the offset of CRC. When you call store.get(storeKeys, storeGetOptions)
, it returns a MessageReadSet
and a list of MessageInfo
. In this case, we only have one MessageInfo
and one MessageReadSet
. The MessageInfo
's getSize()
would return the size of this entire message, what we are trying to get is the crc of this message. As long as we know the message size, we can just do MessageInfo.getSize() - MessageFormatRecord.Crc_Size
. Something like that.
So you don't really need to have a new method to get the crc offset.
ambry-server/src/main/java/com/github/ambry/server/AmbryRequests.java
Outdated
Show resolved
Hide resolved
long crcOffset = msgFmt.getBlobCRCOffset(rdset, 0); // index = 0, as we have 1 msg | ||
rdset.doPrefetch(0, crcOffset, 8); // crc is 8 bytes | ||
long crc = rdset.getPrefetchedData(0).getLong(0); | ||
newMsgInfo = new MessageInfo(crc, minfo); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have to release the prefetched data buffer.
rdset.getPrefetchedData(0).release()
No description provided.