Skip to content

2.27.0.0-b418

@Sumukh-Phalgaonkar Sumukh-Phalgaonkar tagged this 06 Aug 17:55
Summary:
This revision contains changes for registering CDC service on master and calling GetChanges rpc on the sys catalog tablet. This is required to support the PG like semantics for dynamic table addition in logical replication.

Before this revision, cdc service used to run only on tserver. Now we will register cdc service on master, at startup, along with other services like MasterTabletService, PgClientService etc. The cdc service will be enabled on master only when the first CDC stream (replication slot) will be created. What this means is UpdatePeersAndMetrics will start reading the state table and moving the retention barriers on the master only after the first stream has been created.

The flag `master_cdc_svc_queue_length` determines the rpc queue length for CDC service running on master and has default value 50. The flag `master_cdc_semaphore_capacity` determines the maximum number of concurrent GetChanges calls supported by the CDC service running on the master and has a default value of 10 (same as max_replication_slots).

In order to call GetChanges on the sys catalog tablet, we need to set the retention barriers on it. In this revision this is done in a hacky way, where we setup the retention barriers on sys catalog tablet at the time of stream creation only on the master leader (the master peer which services the CreateCDCStream request). This will be done as a RAFT operation in coming revisions.

To accurately determine any change to the publication, be it in the form of `ALTER PUBLICATION` or due to the creation of new tables (in ALL TABLES publications) we need CHANGE records from two catalog tables, namely `pg_publication_rel` and `pg_class`. At the time of stream creation, these tables are added to the appropriate stream metadata data structures.

With this feature, new GetChanges rpcs will be sent to master. The Update Peers and Metrics thread will also start running on the master. To reduce the load on master, a heartbeat based optimisation will be introduced in subsequent diffs, wherein GetChanges will be sent to master only when the `cdc_catalog_version` changes. This will significantly reduce the number of GetChanges calls to master. Also Update Peers and Metrics thread on master will run at a slower frequency and read only the entries corresponding to the sys catalog tablet.

####Upgrade / rollback safety
All the changes for this feature will be guarded by a default false test flag `TEST_ysql_yb_enable_implicit_dynamic_tables_logical_replication` until the development is complete. This flag will then be made a preview flag and after sufficient bake time be promoted to a default true auto flag.

- The change records from catalog tables, will have their column types and values populated in the fields `QLTypePB master_type` and `QLValuePB master_value`, inside the row message.

This feature is not rollback safe.
Jira: DB-15787

Test Plan: ./yb_build.sh --cxx-test integration-tests_cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestPollingPgCatalogTables

Reviewers: skumar, asrinivasan, stiwary, hsunder

Reviewed By: asrinivasan, hsunder

Subscribers: hsunder, ycdcxcluster

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D45445
Assets 2
Loading