Basic ADD COLUMN and DROP COLUMN replay support for PG, BQ and SF #368
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
heavycrystal
force-pushed
the
schema-changes-basic
branch
from
28a37d9
to
f05f390
Compare
iskakaushik
reviewed
Sep 4, 2023
heavycrystal
force-pushed
the
schema-changes-basic
branch
from
71a9670
to
6f55cda
Compare
iskakaushik
changed the title
iskakaushik
added a commit
that referenced
this pull request
Sep 30, 2023
This PR is debugging an issue where schema change propagation (#368) don't apply properly when there are parallel transactions happening during an ALTER TABLE statement. Borrowing from jackc/pglogrepl#59 This PR changes the handling of clientXLogPos to match the one from Postgres' own `pg_recvlogical`: - keepalive messages should bump the position too, as they're only sent (from what I can tell) after any xlogdata message; - both the WALStart and the ServerWALEnd in logical xlogdata messages represent the position that should be reported back, and adding the length of the post-decoding data to it is meaningless; - relation messages have a position of zero, and in general we should match the pg_recvlogical behavior of only increasing the local position. --------- Co-authored-by: Kaushik Iska <iska.kaushik@gmail.com>
iskakaushik
pushed a commit
that referenced
this pull request
Oct 5, 2023
#480) This PR adds on to the existing work towards syncing schema changes in CDC mirrors (#368): 1. handles an edge case where a column is dropped and recreated with a different data type in the same ALTER TABLE statement. 2. Adds quoting, guarding against case-sensitive column names and reserved identifiers. 3. Fixes type mapping for BigQuery Additionally, adds some tests for ReplayTableSchemaDelta for Postgres and Snowflake. Bigquery does not have tests yet but has been manually tested.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.