Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Mysqlcdc2paimon Action Error When Adding Int Column #3501

Closed
1 of 2 tasks
Pandas886 opened this issue Jun 11, 2024 · 1 comment · Fixed by #3506
Closed
1 of 2 tasks

[Bug] Mysqlcdc2paimon Action Error When Adding Int Column #3501

Pandas886 opened this issue Jun 11, 2024 · 1 comment · Fixed by #3506
Labels
bug Something isn't working

Comments

@Pandas886
Copy link
Contributor

Pandas886 commented Jun 11, 2024

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.8

Compute Engine

flink1.18

Minimal reproduce step

During the synchronization process of mysqlcdc2paimon action with database combine mode, an error will occur when adding a new int column in the table in mysql (schema change).

java.lang.UnsupportedOperationException: Don't support MySQL type 'INTEGER' yet.
	at org.apache.paimon.flink.action.cdc.mysql.MySqlTypeUtils.toDataType(MySqlTypeUtils.java:311)
	at org.apache.paimon.flink.action.cdc.mysql.MySqlRecordParser.extractFields(MySqlRecordParser.java:186)
	at org.apache.paimon.flink.action.cdc.mysql.MySqlRecordParser.extractSchemaChange(MySqlRecordParser.java:164)
	at org.apache.paimon.flink.action.cdc.mysql.MySqlRecordParser.flatMap(MySqlRecordParser.java:122)
	at org.apache.paimon.flink.action.cdc.mysql.MySqlRecordParser.flatMap(MySqlRecordParser.java:75)
	at org.apache.flink.streaming.api.operators.StreamFlatMap.processElement(StreamFlatMap.java:47)
	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:108)
	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:77)
	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)
	at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:309)
	at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110)
	at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101)
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:154)
	at org.apache.paimon.flink.action.cdc.serialization.CdcDebeziumDeserializationSchema.deserialize(CdcDebeziumDeserializationSchema.java:78)
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:129)
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:106)
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:83)
	at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55)
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:160)
	at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419)
	at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
	at java.lang.Thread.run(Thread.java:748)

What doesn't meet your expectations?

error happen

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@Pandas886 Pandas886 added the bug Something isn't working label Jun 11, 2024
@MOBIN-F
Copy link
Contributor

MOBIN-F commented Jun 11, 2024

hi,@Pandas886 I submitted a PR that should solve this problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants