[xCluster] Improve IsBootstrapRequired check #18698
Assignees
Labels
area/docdb
YugabyteDB core features
kind/enhancement
This is an enhancement of an existing feature
priority/low
Low priority
Comments
hari90
added
area/docdb
yugabyte-ci
added
kind/enhancement
yugabyte-ci
added
priority/low
lingamsandeep
changed the title
hari90
added a commit
that referenced
this issue
Sep 22, 2023
…e for IsBootstrapRequired check of a new stream Summary: Original commit: 7d821a4 / D28545 When creating a new xcluster stream we first run IsBootstrapRequired to check if the stream needs a bootstrap (WAL checkpoint + backup restore). The current check uses the last WAL index to perform this check and declares that if any operation is found in the WAL then a bootstrap is needed. This makes it safe but it is over aggressive since tablet leader failovers generate NO_OP entries that can incorrect trigger a bootstrap. Also when we add table to replication we change the WAL retention which also generates an entry in the WAL. Other user actions like schema changes, or inserting and deleting data also results in a similar behavior. This commit changes the check to see if there is any visible rows in the tablet. If any rows are present then we will require a bootstrap. This is true even if we have all the WAL since the data could be external ops in the WAL. Since these external WAL ops are not replicated just replicating all the WAL will not be sufficient to replicate existing data. This is common in DR failover and failback scenarios. We use FetchNext call of the RowIterator to check if at least one visible row exists in any of the tables of the tablet. Fixes #18698 Jira: DB-7607 Test Plan: xcluster tests Reviewers: jhe, xCluster Reviewed By: jhe Subscribers: ybase, ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D28680
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-7607
Description
Today we check latest opid is 1 to determine if any WAL entries exist. This is over aggressive as we can have WAL entries that dont add any data like change metadata ops. Also we can have data even without any WAL in cases where we restore a DB.
A better option is to check if there are any rows in the tablet instead of relying on WAL opIDs.
Warning: Please confirm that this issue does not contain any sensitive information
The text was updated successfully, but these errors were encountered: