New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DocDB] PITR - Single file compaction does not reset Hybrid Time filter set during restore #17261
Closed
1 task done
Labels
2.14 Backport Required
2.16 Backport Required
2.18 Backport Required
area/docdb
YugabyteDB core features
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Projects
Comments
sanketkedia
added
area/docdb
YugabyteDB core features
status/awaiting-triage
Issue awaiting triage
labels
May 10, 2023
yugabyte-ci
added
kind/bug
This issue is a bug
priority/medium
Medium priority issue
labels
May 10, 2023
sanketkedia
added a commit
that referenced
this issue
May 12, 2023
…e filter Summary: If there's only one input file for compaction then the output file generated by compaction still contains the hybrid time filter. It is not reset. This is because the largest and smallest frontiers are cloned as is from the first input file and then iterating over the subsequent input files, we update them. During update, we reset the filter. So if there's only one input file the filter is not reset. This is a perf fix rather than correctness. If we have this filter then we go through the hassle of creating a filtering iterator wrapper over the regular rocksdb iterator and comparing every key against it which is not necessary given that there are no records in the file that will match Jira: DB-6496 Test Plan: ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/0 ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/1 Reviewers: bogdan, sergei Reviewed By: sergei Subscribers: zdrudi, mhaddad, slingam, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D25216
sanketkedia
added a commit
that referenced
this issue
Jun 13, 2023
…eset Hybrid Time filter Summary: Original commit: 639133f / D25216 If there's only one input file for compaction then the output file generated by compaction still contains the hybrid time filter. It is not reset. This is because the largest and smallest frontiers are cloned as is from the first input file and then iterating over the subsequent input files, we update them. During update, we reset the filter. So if there's only one input file the filter is not reset. This is a perf fix rather than correctness. If we have this filter then we go through the hassle of creating a filtering iterator wrapper over the regular rocksdb iterator and comparing every key against it which is not necessary given that there are no records in the file that will match Jira: DB-6496 Test Plan: ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/0 ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/1 Reviewers: sergei, bogdan Reviewed By: bogdan Subscribers: ybase, slingam, mhaddad, zdrudi Differential Revision: https://phorge.dev.yugabyte.com/D26093
sanketkedia
added a commit
that referenced
this issue
Jun 13, 2023
…eset Hybrid Time filter Summary: Original commit: 639133f / D25216 If there's only one input file for compaction then the output file generated by compaction still contains the hybrid time filter. It is not reset. This is because the largest and smallest frontiers are cloned as is from the first input file and then iterating over the subsequent input files, we update them. During update, we reset the filter. So if there's only one input file the filter is not reset. This is a perf fix rather than correctness. If we have this filter then we go through the hassle of creating a filtering iterator wrapper over the regular rocksdb iterator and comparing every key against it which is not necessary given that there are no records in the file that will match Jira: DB-6496 Test Plan: ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/0 ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/1 Reviewers: sergei, bogdan Reviewed By: bogdan Subscribers: zdrudi, mhaddad, slingam, ybase Differential Revision: https://phorge.dev.yugabyte.com/D26092
sanketkedia
added a commit
that referenced
this issue
Jun 13, 2023
…eset Hybrid Time filter Summary: Original commit: 639133f / D25216 If there's only one input file for compaction then the output file generated by compaction still contains the hybrid time filter. It is not reset. This is because the largest and smallest frontiers are cloned as is from the first input file and then iterating over the subsequent input files, we update them. During update, we reset the filter. So if there's only one input file the filter is not reset. This is a perf fix rather than correctness. If we have this filter then we go through the hassle of creating a filtering iterator wrapper over the regular rocksdb iterator and comparing every key against it which is not necessary given that there are no records in the file that will match Jira: DB-6496 Test Plan: ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/0 ybd release --cxx_test docdb-test --gtest-filter DocDBTests/DocDBTestWrapper.SetHybridTimeFilterSingleFile/1 Reviewers: sergei, bogdan Reviewed By: bogdan Subscribers: ybase, slingam, mhaddad, zdrudi Differential Revision: https://phorge.dev.yugabyte.com/D26091
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
2.14 Backport Required
2.16 Backport Required
2.18 Backport Required
area/docdb
YugabyteDB core features
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Jira Link: DB-6496
Description
If there's only one input file for compaction then the output file generated by compaction still contains the hybrid time filter. It is not reset. This is because the largest and smallest frontiers are cloned as is from the first input file and then iterating over the subsequent input files, we update them. During update, we reset the filter. So if there's only one input file the filter is not reset.
Warning: Please confirm that this issue does not contain any sensitive information
The text was updated successfully, but these errors were encountered: