RATIS-1723. CounterStateMachine should update the latest snapshot.#765
RATIS-1723. CounterStateMachine should update the latest snapshot.#765codings-dan merged 3 commits intoapache:masterfrom
Conversation
| private volatile File stateMachineDir = null; | ||
|
|
||
| private volatile SingleFileSnapshotInfo currentSnapshot = null; | ||
| private final AtomicReference<SnapshotInfo> latestSnapshot = new AtomicReference<>(); |
There was a problem hiding this comment.
If we use AtomicReference<SingleFileSnapshotInfo> here, we don't have to cast in the caller site if we're sure that the snapshot storage is SimpleStateMachineStorage.
There was a problem hiding this comment.
This is a good idea. Thanks!
| } | ||
|
|
||
| // update storage | ||
| final MD5Hash md5 = MD5FileUtil.computeAndSaveMd5ForFile(snapshotFile); |
There was a problem hiding this comment.
Is there any reader of the saved md5 file? Does the file have an owner that remove it on exiting/snapshot cleanup?
There was a problem hiding this comment.
We should verify the md5 when loading the snapshot.
The md5 file should be removed when the snapshot file is removed.
There was a problem hiding this comment.
should be removed
Can you show me the code path to implement this clean-up logic? I don't find it :(
There was a problem hiding this comment.
Currently, we don't have clean-up logic implemented in the examples.
There was a problem hiding this comment.
@tisonkun , actually, the old snapshots are deleted in StateMachineUpdater.takeSnapshot() according to the SnapshotRetentionPolicy. It should also delete the md5 files.
tisonkun
left a comment
tisonkun
left a comment
There was a problem hiding this comment.
LGTM.
I ask some questions about cleaning up MD5 files because I ever saw Alibaba's celeborn project write its own clean up logics to do so and I wonder if it can be some code in the upstream :)
Thanks for the info! Good to know. |
codings-dan
left a comment
codings-dan
left a comment
There was a problem hiding this comment.
the change looks good, +1
|
@codings-dan , thanks a lot for reviewing this! |
) (cherry picked from commit b8117f4)
…pache#765) * RATIS-1723. CounterStateMachine should update the latest snapshot. * Address review comments and fix bugs. * delete md5 file
3 participants
See https://issues.apache.org/jira/browse/RATIS-1723