Prevent deletion of replication slot entry on table drop #22117
Assignees
Labels
Comments
yugabyte-ci
added
area/cdcsdk
siddharth2411
added a commit
that referenced
this issue
Apr 24, 2024
…drop Summary: Whenever a table is dropped that is part of the CDC stream, `CleanUpCDCSDKStreamsMetadata()` is called to remove the cdc state table entries and sys catalog entries. CleanUpCDCSDKStreamsMetadata computes tablets from two sources: Set A - GetTablets() on all tables part of the stream metadata. Set B - Read cdc_state table for the stream. Remove entries from B that are not present in A. We recently introduced a new state table entry for the replication slot. So, based on the above algorithm, it is deleted from the cdc state table whenever a table is dropped from that stream. To stop this deletion, we are simply not considering the slot entry in the above algorithm. Jira: DB-11044 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* ./yb_build.sh --cxx-test cdcsdk_consumption_consistent_changes-test --gtest_filter CDCSDKConsumptionConsistentChangesTest.TestConsumptionAfterDroppingTableNotInPublication Reviewers: asrinivasan, stiwary Reviewed By: asrinivasan Subscribers: ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D34423
siddharth2411
added a commit
that referenced
this issue
Apr 25, 2024
…t entry on table drop Summary: Original commit: 23dde81 / D34423 Whenever a table is dropped that is part of the CDC stream, `CleanUpCDCSDKStreamsMetadata()` is called to remove the cdc state table entries and sys catalog entries. CleanUpCDCSDKStreamsMetadata computes tablets from two sources: Set A - GetTablets() on all tables part of the stream metadata. Set B - Read cdc_state table for the stream. Remove entries from B that are not present in A. We recently introduced a new state table entry for the replication slot. So, based on the above algorithm, it is deleted from the cdc state table whenever a table is dropped from that stream. To stop this deletion, we are simply not considering the slot entry in the above algorithm. Jira: DB-11044 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* ./yb_build.sh --cxx-test cdcsdk_consumption_consistent_changes-test --gtest_filter CDCSDKConsumptionConsistentChangesTest.TestConsumptionAfterDroppingTableNotInPublication Reviewers: asrinivasan, stiwary, xCluster, hsunder Reviewed By: stiwary Subscribers: ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D34534
svarnau
pushed a commit
that referenced
this issue
May 25, 2024
…drop Summary: Whenever a table is dropped that is part of the CDC stream, `CleanUpCDCSDKStreamsMetadata()` is called to remove the cdc state table entries and sys catalog entries. CleanUpCDCSDKStreamsMetadata computes tablets from two sources: Set A - GetTablets() on all tables part of the stream metadata. Set B - Read cdc_state table for the stream. Remove entries from B that are not present in A. We recently introduced a new state table entry for the replication slot. So, based on the above algorithm, it is deleted from the cdc state table whenever a table is dropped from that stream. To stop this deletion, we are simply not considering the slot entry in the above algorithm. Jira: DB-11044 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* ./yb_build.sh --cxx-test cdcsdk_consumption_consistent_changes-test --gtest_filter CDCSDKConsumptionConsistentChangesTest.TestConsumptionAfterDroppingTableNotInPublication Reviewers: asrinivasan, stiwary Reviewed By: asrinivasan Subscribers: ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D34423
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-11044
The text was updated successfully, but these errors were encountered: