2.25.0.0-b21
tagged this
20 Sep 20:03
Summary: Previously, when a table rewrite DDL operation (such as `ALTER COLUMN TYPE`, `ADD/DROP PRIMARY KEY` or `TRUNCATE TABLE`) was performed before creating a replication slot, the changes from that table were not streamed during logical replication. The issue arose from the inconsistent handling of DocDB's UUID for a table's ID and PG’s `table_oid`. Normally, the table’s UUID in DocDB matches the `table_oid` in PG. However, in the case of table rewrites, while PG’s `table_oid` remains the same, the DocDB UUID changes. After such rewrites, the `relfilenode` in PG becomes the one-to-one mapping to the new UUID in DocDB (commit: bb77fb2052ad259457b01c586dc96075ad88b958). For clarity, the following terminology will be used throughout this summary: - `table_id`: Refers to the UUID of the underlying DocDB table. - `pg_table_id`: Refers to the original UUID of the DocDB table before the table rewrite. - `table_oid`: Represents PG’s table OID. - `relfilenode`: Corresponds to the relfilenode of the PG table, which maps to DocDB’s UUID. It’s important to note that without any table rewrites, `table_oid` is equal to `relfilenode`. However, with table rewrites, `relfilenode` reflects the new underlying table, while `table_oid` remains unchanged. The following key changes were made to address the issue in this revision: 1. In `YBCInitVirtualWalForCDC`, `table_id` is now computed using `relfilenode` instead of `table_oid` to ensure the correct `table_id` is passed to CDC Service. 2. The `GetConsistentChanges` response now treats the `table_id` as `relfilenode` and accurately retrieves `table_oid` to be used within PG. Previously, `table_id` was directly treated as `table_oid`, causing inconsistencies. 3. Similarly, when fetching the `replica_identity_map` (which maps table_id to its replica identity), the `table_id` was previously used directly to obtain the `table_oid`. Now, it is treated as `relfilenode`, and the `table_oid` is retrieved accordingly. JIRA: DB-12672 Test Plan: == Automated Testing == ./yb_build.sh --java-test org.yb.pgsql.TestPgReplicationSlot#testWithAlterType ./yb_build.sh --java-test org.yb.pgsql.TestPgReplicationSlot#testWithDropAddPk ./yb_build.sh --java-test org.yb.pgsql.TestPgReplicationSlot#testWithTruncate Reviewers: skumar, sumukh.phalgaonkar, stiwary, siddharth.shah, asrinivasan, xCluster, hsunder Reviewed By: skumar, stiwary Subscribers: yql, ybase, ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D37721