TryCatchUpWithPrimary: skip MANIFEST tail reads when file size did not change#14124
Open
dungeon-master-666 wants to merge 1 commit intofacebook:mainfrom
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I noticed that
TryCatchUpWithPrimarytakes a lot of time in my setup even when there were no changes in DB. In my case for 500 GB RocksDB instance it takes up to ~50 ms and most of this time is tail reading of MANIFEST file.In this PR I implement the following optimization - to cache MANIFEST file size and if it did not grow since last call immediately return control. It makes sense for 2 reasons:
Implementation details:
ReactiveVersionSetto query the currentMANIFESTsize via the filesystemVersionSet::manifest_file_size_and skipManifestTailer::Iterate()when the file hasn’t grownThis optimization speeds up
TryCatchUpWithPrimaryup to 3 times in my tests.