-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make secondary instance use ManifestTailer #7998
Closed
Closed
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
1cce11f
Add new class ManifestTailer
riversand963 141cde0
Use ManifestTailer in ReactiveVersionSet
riversand963 c823ed6
Update tests
riversand963 85f180f
Add a test for switching MANIFEST twice
riversand963 ce64012
Update the signature of ReadAndApply a bit
riversand963 3e5a127
Address comments
riversand963 7c0405f
Disable flaky SwitchWAL unit test
riversand963 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In which condition, will it reach here? Should not new CF only be handled in Recovery mode (which is handled by
VersionEditHanlder::OnColumnFamilyAdd()
)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the primary later creates a new column family, it will write a VersionEdit to the MANIFEST. The secondary, whether in Recovery mode or Catchup mode, will be able to see this VersionEdit.
If the column family is added by the primary after the secondary is created and the secondary does not specify it in its
Open
call, then we ignore it. Should the secondary be interested in this column family, it should be opened with this column family in the first place. This is aligned with how we callDB::Open()
with column families.For column families that are specified when creating the secondary, we will see
VersionEdit.is_column_family_add == true
for them multiple times, whether secondary is in Recovery or Catchup mode. In fact, the primary will write a VersionEdit to add these column families each time it switches to a new MANIFEST.Maybe an example can help.
The code snippet here handles the case when the
add_cf_1
in MANIFEST-4 is seen. Suppose at this time, the secondary instance sees the second version edit of MANIFEST-1 but for some reason switches to MANIFEST-4. We need to build the version storage for cf1 from the empty base version of cf1, but from the version with 2.sst, otherwise the consistency check will fail due to two occurences of 2.sst.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the explanation.