Summary:
Adding in support for non-colocated and non-table rewrite ALTER TABLE commands.
There are 2 main types of schemas in YB:
1. PG schema, which is stored in PG catalog
2. docdb schema, the current docdb view of the table, stored in system catalog on master
There is also a packing schema stored in each TabletMetadata which is used to read/write rows, this is based off of the docdb schema versions. The packing schema also stores historical packing schemas in order to read older rows.
XCluster maintains a mapping of source to compatible target packing schema versions. This is used to rewrite the packing version of replicated records.
When an ALTER TABLE is run, it can create multiple ChangeMetadataOps (CMOP) that change some part of the schema. Previously, xCluster would pause on each CMOP until it found a compatible packing version on the target (usually triggered by a user running the DDL again on the target).
For DDL replication, we cannot pause on CMOPs since otherwise we will run into a deadlock with the ddl_queue poller (since the ddl_queue poller needs to be the last one to update safe time, it waits for all other pollers to catch up to its safe time before running DDLs). Thus we need some way of processing CMOPs and new rows before the DDL is actually run. Note that since we only read at xCluster safe time, we won't see any of these new rows until the DDL is run by ddl_queue.
In order to process these CMOPs, this diff introduces `InsertPackedSchemaForXClusterTarget`. This is an additional field provided to AlterTable requests that will insert a new historical packed schema. However, since packed schema versions are based on docdb schema versions, we need to do the following:
1. Suppose we have docdb schema S with version N, and want to insert a new packed schema S'
2. We will bump up the docdb schema version to N+2 and keep the schema as S
3. Send an AlterTable request to tablets with `insert_packed_schema`
4. Tablets will update their packing schemas, such that it will now include [(N -> S), (N+1 -> S'), (N+2 -> S)]
Thus in the end, we still have the same schema, but also successfully have a new historical packing schema with this new schema.
With `InsertPackedSchemaForXClusterTarget`, xCluster can now call this function when receiving a CMOP with a new packing schema. Pollers can then continue polling and use this new packing schema version for their rewrites.
Note that the docdb schema and pg schema are still both in sync, and match what the expected schemas are at the current xcluster safe time. Then once the ddl_queue replicates the ALTER TABLE, this DDL is run as normal, which will update pg and docdb schemas as desired.
**Upgrade/Rollback safety:**
Only adding in new fields/pbs that are blocked behind automatic ddl replication test flags
---
A few follow ups not addressed in this diff:
1. Ensure that ADD COLUMN doesn't waste column ids on failures, since we require that when the DDL is replicated, it will produce the same packing schema as from the source.
2. (#23078) Skip table schema verification for automatic replication, and instead use the table id to find and link streams. Currently causes issues if a table is created then immediately renamed/altered.
3. (#22318) Support for colocated relations, more complicated there since we can get CMOPs without the table even existing yet.
Jira: DB-12850
Test Plan:
```
ybd --cxx-test xcluster_ddl_replication-test --gtest_filter "XClusterDDLReplicationAddDropColumnTest.AddDropColumns"
ybd --cxx-test xcluster_ddl_replication_pgregress-test --gtest_filter "XClusterPgRegressDDLReplicationTest.PgRegressAlterTable"
```
Reviewers: hsunder, xCluster
Reviewed By: hsunder
Subscribers: yql, ybase
Differential Revision: https://phorge.dev.yugabyte.com/D38293