-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature][Kafka] Support multi-table source read #5992
[Feature][Kafka] Support multi-table source read #5992
Conversation
issues: #5677 |
# Conflicts: # seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/RestHttpPostCommandProcessor.java
# Conflicts: # docs/en/connector-v2/source/kafka.md
please resolve conflict |
# Conflicts: # seatunnel-formats/seatunnel-format-compatible-connect-json/src/main/java/org/apache/seatunnel/format/compatible/kafka/connect/json/CompatibleKafkaConnectDeserializationSchema.java # seatunnel-formats/seatunnel-format-json/src/test/java/org/apache/seatunnel/format/json/JsonRowDataSerDeSchemaTest.java
public DebeziumJsonDeserializationSchema( | ||
SeaTunnelRowType rowType, | ||
boolean ignoreParseErrors, | ||
boolean debeziumEnabledSchema, | ||
CatalogTable catalogTable) { | ||
this.rowType = rowType; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public DebeziumJsonDeserializationSchema( | |
SeaTunnelRowType rowType, | |
boolean ignoreParseErrors, | |
boolean debeziumEnabledSchema, | |
CatalogTable catalogTable) { | |
this.rowType = rowType; | |
public DebeziumJsonDeserializationSchema( | |
CatalogTable catalogTable, | |
boolean ignoreParseErrors, | |
boolean debeziumEnabledSchema) { | |
this.rowType = catalogTable.getSeaTunnelRowType(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update all places
…feature-multi-table-kafka # Conflicts: # seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceConfig.java # seatunnel-formats/seatunnel-format-compatible-connect-json/src/main/java/org/apache/seatunnel/format/compatible/kafka/connect/json/CompatibleKafkaConnectDeserializationSchema.java
|
||
public AvroDeserializationSchema(SeaTunnelRowType rowType) { | ||
public AvroDeserializationSchema(SeaTunnelRowType rowType, CatalogTable catalogTable) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public AvroDeserializationSchema(SeaTunnelRowType rowType, CatalogTable catalogTable) { | |
public AvroDeserializationSchema(CatalogTable catalogTable) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Purpose of this pull request
Support for kafka multiple data sources
Does this PR introduce any user-facing change?
The same as before
How was this patch tested?
e2e Adds multi-source tests including TOPIC Regex matching
Check list
New License Guide
release-note
.