-
Notifications
You must be signed in to change notification settings - Fork 296
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Enable data preload write through #189
Conversation
@@ -237,7 +237,7 @@ void CloudMergeTreeBlockOutputStream::writeSuffixImpl() | |||
{ | |||
/// auto testlog = std::make_shared<TestLog>(const_cast<Context &>(context)); | |||
/// TEST_START(testlog); | |||
/// tryPreloadChecksumsAndPrimaryIndex(storage, std::move(preload_parts), ManipulationType::Insert, context); | |||
tryPreload(context, storage, preload_parts); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to support finish the preload before parts become visible?
@@ -483,4 +484,37 @@ void CnchDataWriter::publishStagedParts( | |||
commitDumpedParts(items); | |||
} | |||
|
|||
void tryPreload( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does mergemutate task preload the part?
5b5d638
to
f4778d7
Compare
/// tryPreloadChecksumsAndPrimaryIndex(storage, std::move(preload_parts), ManipulationType::Insert, context); | ||
/// TEST_END(testlog, "Finish tryPreloadChecksumsAndPrimaryIndex in batch mode"); | ||
} | ||
cnch_writer.preload(preload_parts); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it duplicated? since it has been preloaded before commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where do you refer to the previous preload
? here it's actually preload after commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the preload is also called such as commit part.
Changelog category :
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Write through cache strategy,
add new settings to enable writing to disk cache while inserting data
...
Documentation entry for user-facing changes