New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inserting only non-duplicate chunks in MV #54184
Inserting only non-duplicate chunks in MV #54184
Conversation
Technical description, feel free to comment or make any follow-up question. This PR aims to fix the following case: when data is inserted in a replicated table, it may not be inserted in an associated MV if some of it (but not all) is duplicated. As a quick example, if we have a table
Currently neither the case The approach followed in order to solve this has been to change the behavior of the class As a bit of simplification due to these changes, I removed the method |
This is an automated comment for commit ec1f7b3 with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page Successful checks
|
Interesting, probably it was the cause of lot of issues with the inconsistency between MatView and source table. |
Thanks to the test |
After checking, just one test failed Same test ran successfully in other suites: asan-analyzer, release, tsan... I checked the logs and the errors don't seem related to sanitizing at all, instead there are exceptions of the kind: |
std::unique_ptr<IOutputChunkGenerator> IOutputChunkGenerator::createCopyRanges(ContextPtr context) | ||
{ | ||
// If MV is responsible for deduplication, block is not considered duplicated. | ||
if (context->getSettingsRef().deduplicate_blocks_in_dependent_materialized_views) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about checking this outside? so that we do not have to pass context
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fixed error that prevented insertion in replicated MV of new data in presence of duplicated data