DBZ-49 MySQL DDL parser should be more tolerant of REFERENCE clauses in CREATE TABLE statements #42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…TE TABLE statements MySQL 5.6 using the MyISAM engine will create the `help_relation` system table using a CREATE TABLE statement that does not have in the columns' REFERENCE clause a list of columns in the referenced table. MySQL 5.7 using the InnoDB engine does not include the REFERENCE clauses. Because Debezium's MySQL DDL parser is meant only to understand the statements recorded in the binlog, it does not have to validate the statements and therefore the DDL parser can be a bit more lenient by not requiring the list of columns in a REFERENCE clause in a CREATE TABLE statement's column definitions. This commit also adds several unit tests that validate all of the DDL statements used by MySQL 5.6 and 5.7 during startup (in the configurations used in our integration tests).
rhauch
changed the title
Closed
xinbinhuang
pushed a commit
to xinbinhuang/debezium
that referenced
this pull request
Jun 27, 2023
r? ywu shichao [CDC-683](https://jira.corp.stripe.com/browse/CDC-683) Test payload: ([full output gist](https://git.corp.stripe.com/gist/binh/be9605c903f02c127fa5240920f5b1f9)) - Move the stripe audit field parsing into sourceInfo - Then use that to add backfill filtering on change stream - I left the oplog code intact cuz, don’t wanna touch it and we’ll deprecate soon.? ``` { ... other fields "payload": { "before": "LQAAAAJfaWQADAAAAGlkX3NsY190ZXN0ABJ0aW1lc3RhbXAATMNTVkefQhcA", "after": "LQAAAAJfaWQADAAAAGlkX3NsY190ZXN0ABJ0aW1lc3RhbXAAAkIb0kefQhcA", "patch": null, "filter": null, "updateDescription": null, "source": { "version": "1.9.0-SNAPSHOT", "connector": "mongodb", "name": "cdc-test", "ts_ms": 1676077142000, "snapshot": "false", "db": "slc_monitor_shard_test_4_4", "sequence": null, "rs": "shard_test_4_4", "collection": "availability_check", "ord": 4, "h": null, "tord": null, "stxnid": null, "lsid": null, "txnNumber": null, "wallTime": null, "stripeAudit": "{\"host\":\"qa-mongohealth--098e2c0b0b217179c.northwest.stripe.io\"}" }, "op": "u", "ts_ms": 1676077142202, "transaction": null, "_raw_oplog": null } } ``` (Squashed by Merge Queue - Original PR: https://git.corp.stripe.com/stripe-private-oss-forks/debezium/pull/42)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
MySQL 5.6 using the MyISAM engine will create the
help_relationsystem table using a CREATE TABLE statement that does not have in the columns' REFERENCE clause a list of columns in the referenced table. MySQL 5.7 using the InnoDB engine does not include the REFERENCE clauses.Because Debezium's MySQL DDL parser is meant only to understand the statements recorded in the binlog, it does not have to validate the statements and therefore the DDL parser can be a bit more lenient by not requiring the list of columns in a REFERENCE clause in a CREATE TABLE statement's column definitions.
This commit also adds several unit tests that validate all of the DDL statements used by MySQL 5.6 and 5.7 during startup (in the configurations used in our integration tests).