[CDCSDK] Persist the safe_time from explicit checkpoint in cdc_state table #18116
Assignees
Labels
2.18 Backport Required
2.18.0_blocker
area/cdcsdk
CDC SDK
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Projects
Comments
yugabyte-ci
added
kind/bug
adithya-kb
added
2.18 Backport Required
2.18.0_blocker
and removed
kind/bug
adithya-kb
pushed a commit
that referenced
this issue
Jul 7, 2023
…c_state table Summary: We need to start persisting the "safe_time" (currently passed through the snapshot_time field) which is passed by the connector through the "explicit_cdc_sdk_checkpoint" field of GetChangesRequestPB. Previously, we only were storing the safe_time when either BEFORE_IMAGE was enabled or when we were in snapshot mode, and we could directly pass the stored value to the tablet_peers to stop compaction. Now since the safe_time is going to be stored always , we need changes in "UpdatePeersAndMetrics", specifically in the method: "PopulateTabletCheckPointInfo", which populates the safe_time to be passed to the tablet_peers. We will now add checks to populate the safe_time to be propogated, only when before image is enabled or when a snapshot_key is present i.e when in snapshot mode. Jira: DB-7149 Test Plan: Jenkins: skip ybd --cxx-test integration-tests_cdcsdk_before_image-test Reviewers: skumar, vkushwaha Reviewed By: skumar Subscribers: ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D26701
adithya-kb
pushed a commit
that referenced
this issue
Jul 12, 2023
…heckpoint in cdc_state table Summary: Original commit: 82ef76c / D26701 We need to start persisting the "safe_time" (currently passed through the snapshot_time field) which is passed by the connector through the "explicit_cdc_sdk_checkpoint" field of GetChangesRequestPB. Previously, we only were storing the safe_time when either BEFORE_IMAGE was enabled or when we were in snapshot mode, and we could directly pass the stored value to the tablet_peers to stop compaction. Now since the safe_time is going to be stored always , we need changes in "UpdatePeersAndMetrics", specifically in the method: "PopulateTabletCheckPointInfo", which populates the safe_time to be passed to the tablet_peers. We will now add checks to populate the safe_time to be propogated, only when before image is enabled or when a snapshot_key is present i.e when in snapshot mode. Jira: DB-7149 Test Plan: ybd --cxx-test integration-tests_cdcsdk_before_image-test Reviewers: skumar, vkushwaha, aagrawal Reviewed By: aagrawal Subscribers: ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D26859
yugabyte-ci
added
kind/bug
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-7149
We need to start persisting the "safe_time" (currently passed through the snapshot_time field) which is passed by the connector through the "explicit_cdc_sdk_checkpoint" field of GetChangesRequestPB.
Previously, we only were storing the safe_time when either BEFORE_IMAGE was enabled or when we were in snapshot mode, and we could directly pass the stored value to the tablet_peers to stop compaction. Now since the safe_time is going to be stored always , we need changes in "UpdatePeersAndMetrics", specifically in the method: "PopulateTabletCheckPointInfo", which populates the safe_time to be passed to the tablet_peers. We will now add checks to populate the safe_time to be propogated, only when before image is enabled or when a snapshot_key is present i.e when in snapshot mode.
The text was updated successfully, but these errors were encountered: