Skip to content

2.29.0.0-b148

2.29.0.0-b148: [#28840] CDC: Children tablets to not inherit parent tablet's checkpoint
@devansh5398 devansh5398 tagged this 06 Nov 15:02 
Summary:
Consider the scenario where a tablet split had occurred. The polling of the child tablets were started. However, the update to cdc state table for one of the child tablet didn't persisted.
- As VWAL restarts, it finds the tablets to be polled. Since the cdc state table entry for one of the child tablets didn't succeeded, the parent tablet is returned to be polled.
- VWAL get the tablet split error for GetChanges call and so gets the children tablets to be polled.
- It updates its internal maps for children tablets using the map entry of parent tablet. This wrongly sets the old checkpoint for already polled child tablet.
- If the intents corresponding to records from old checkpoint are already GC'ed, the GetChanges call returns "Trying to fetch already GCed intents" error.

To mitigate this issue, this diff:
- correctly updates VWAL's internal map `tablet_next_req_map_` for children tablets using the information it gets from `CDCServiceImpl::GetTabletListToPollForCDC` instead of using their parent's entry info.
- removes the logic of populating children tablets entry in `tablet_last_sent_req_map_` and `commit_meta_and_last_req_map_`. This way it prevents any scenario which possibly can move explicit checkpoint of a child tablet backwards.
- sets the safe time in GetChanges request to be `max` of the `next_req_info's safe_hybrid_time` and `last persisted restart time` because upon restart VWAL will always send records with commit time greater than restart time. Hence, it is safe and also optimized to do so.

Jira: DB-18557

Test Plan:
yb_build.sh --cxx-test cdcsdk_consumption_consistent_changes-test --gtest_filter=CDCSDKConsumptionConsistentChangesTest.TestChildTabletPolledFromLatestCheckpointOnVWALRestart
yb_build.sh --cxx-test cdcsdk_consumption_consistent_changes-test --gtest_filter=CDCSDKConsumptionConsistentChangesTest.TestChildrenTabletsCheckpointMoveAheadOfParentTablet

Reviewers: sumukh.phalgaonkar, asrinivasan, skumar, stiwary

Reviewed By: sumukh.phalgaonkar, asrinivasan

Subscribers: ycdcxcluster

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D47783
Assets 2
Loading