Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Failed when a table that was not included in the source changed its schema. #2981

Closed
1 of 2 tasks
lvyanquan opened this issue Jan 9, 2024 · 0 comments · Fixed by #2986
Closed
1 of 2 tasks
Labels
bug Something isn't working

Comments

@lvyanquan
Copy link
Contributor

lvyanquan commented Jan 9, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Flink version

1.18

Flink CDC version

3.0

Database and its version

Mysql

Minimal reproduce step

run pipeline job with the following file, capture only one inventory.geom table:

################################################################################
# Description: Sync MySQL one table to Values
################################################################################
source:
  type: mysql
  hostname: localhost
  port: 3306
  username: root
  password: 123456
  tables: inventory.geom
  server-id: 5400-5404
  server-time-zone: Asia/Shanghai

sink:
  type: values

pipeline:
  name: Sync MySQL Database to Values
  parallelism: 2

Then change a table schema of other tables, like adding a new column for inventory.orders.

What did you expect to see?

Nothing happen.

What did you see instead?

Got error.

Caused by: java.lang.IllegalArgumentException: Unable to apply SchemaChangeEvent for table "inventory.orders" without existing schema
	at com.ververica.cdc.common.utils.Preconditions.checkArgument(Preconditions.java:128) ~[classes/:?]
	at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaManager.applySchemaChange(SchemaManager.java:105) ~[classes/:?]
	at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistryRequestHandler.handleSchemaChangeRequest(SchemaRegistryRequestHandler.java:102) ~[classes/:?]
	at com.ververica.cdc.runtime.operators.schema.coordinator.SchemaRegistry.handleCoordinationRequest(SchemaRegistry.java:157) ~[classes/:?]
	at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.deliverCoordinationRequestToCoordinator(DefaultOperatorCoordinatorHandler.java:143) ~[flink-runtime-1.18.0.jar:1.18.0]

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@lvyanquan lvyanquan added the bug Something isn't working label Jan 9, 2024
@lvyanquan lvyanquan changed the title [Bug] A table structure change occurred in a table that was not captured in the source [Bug] Failed when a table that was not included in the source changed its schema. Jan 10, 2024
leonardBang pushed a commit that referenced this issue Jan 15, 2024
joyCurry30 pushed a commit to joyCurry30/flink-cdc-connectors that referenced this issue Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant