Summary:
In logical replication, we are introducing a new mode that provides parallel consumption of changes from a table using multiple replication slots where each slot is polling changes from a subset of tablets. This model in guarded under a tserver preview flag - `ysql_yb_enable_consistent_replication_from_hash_range`
To support this mode, we have introduced a new slot option "hash_range" that can be passed with the START_REPLICATION SLOT command. This option takes in 2 comma-separated values 'start hash range' & 'end hash range'. Tablets of the particular table whose start hash range falls within the slot's hash range will be considered for streaming data from that slot.
Requirements for this mode:
1. Publication should only contain 1 table.
2. Slot, once started with a particular hash range, can only be restarted with the same hash range.
When the replication slot is started with hash ranges for the 1st time, these values are extracted and the hash_range option is removed from the option list as this list is passed to output plugin for further validation. These extracted values are passed to VirtualWAL. Here, we persist these hash ranges in the slot's cdc_state entry during initialistion of VirtualWAL. These values will be later checked when the slot is restarted. If the slot is started with no hash ranges for the 1st time and thereafter, nothing will be persisted in slot's state table entry and all tablets of tables under the publication will be polled.
**Upgrade/Rollback safety:**
All changes for this model is covered under a new tserver preview flag `ysql_yb_enable_consistent_replication_from_hash_range`. New proto fields introduced in `InitvirtualWALRequestPB` are only populated & accessed if the flag is set. A separate auto flag is not required for proto changes as the RPC is sent from pg to local tserver.
Jira: DB-15209
Test Plan:
Jenkins: urgent
./yb_build.sh --java-test 'org.yb.pgsql.TestPgReplicationSlot#testConsumptionOnSubsetOfTabletsFromMultipleSlots'
./yb_build.sh --java-test 'org.yb.pgsql.TestPgReplicationSlot#testNonNumericHashRangeWithSlot'
./yb_build.sh --java-test 'org.yb.pgsql.TestPgReplicationSlot#testOutOfBoundHashRangeWithSlot'
./yb_build.sh --cxx-test integration-tests_cdcsdk_consumption_consistent_changes-test --gtest_filter CDCSDKConsumptionConsistentChangesTest.TestReplicationWithHashRangeConstraintsAndTabletSplit
Reviewers: skumar, sumukh.phalgaonkar, stiwary, utkarsh.munjal
Reviewed By: sumukh.phalgaonkar, stiwary, utkarsh.munjal
Subscribers: yql, ybase, ycdcxcluster
Tags: #jenkins-ready
Differential Revision: https://phorge.dev.yugabyte.com/D41520