Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mysql -> doris Flink 实时同步 出错 #162

Closed
baisui1981 opened this issue Nov 12, 2022 · 1 comment
Closed

mysql -> doris Flink 实时同步 出错 #162

baisui1981 opened this issue Nov 12, 2022 · 1 comment
Labels
3.6.0-alpha bug Something isn't working

Comments

@baisui1981
Copy link
Member

mysql -> doris flink 通过使用order2 库中

-- order2
update totalpayinfo set last_ver=last_ver+2, modify_time= unix_timestamp( now()) where totalpay_id = '991807005451d719015451d87157003d';

-- doris query
select last_ver,modify_time from totalpayinfo where totalpay_id = '991807005451d719015451d87157003d'\G;
====================Dirty Data=====================
DirtyDataEntry[jobId='1ba9717c0607605634dbd0dc0b46e2c1', jobName='mysql_doris', operatorName='Sink: totalpayinfo', dirtyContent='{"arity":37,"rowKind":"INSERT"}', errorMessage='com.dtstack.chunjun.throwable.WriteRecordException: 
java.lang.ClassCastException: java.lang.Byte cannot be cast to java.lang.Short
	at com.dtstack.chunjun.connector.doris.sink.DorisHttpOutputFormat.writeSingleRecordInternal(DorisHttpOutputFormat.java:91)
	at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:482)
	at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:287)
	at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:96)
	at com.dtstack.chunjun.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:119)
	at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
	at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:39)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317)
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411)
	at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:142)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)
Caused by: java.lang.ClassCastException: java.lang.Byte cannot be cast to java.lang.Short
	at org.apache.flink.table.data.GenericRowData.getShort(GenericRowData.java:144)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter$1.tinyIntType(TISDorisColumnConverter.java:223)
	at com.qlangtech.tis.plugin.ds.DataType.accept(DataType.java:111)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter.lambda$getSerializationConverter$187873ae$1(TISDorisColumnConverter.java:163)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter.lambda$wrapNullableExternalConverter$6e5a4709$1(TISDorisColumnConverter.java:149)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter.lambda$wrapNullableExternalConverter$6e5a4709$1(TISDorisColumnConverter.java:149)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter.toExternal(TISDorisColumnConverter.java:122)
	at com.qlangtech.plugins.incr.flink.chunjun.doris.sink.TISDorisColumnConverter.toExternal(TISDorisColumnConverter.java:45)
	at com.dtstack.chunjun.connector.doris.rest.DorisLoadClient.createInsertVals(DorisLoadClient.java:197)
	at com.dtstack.chunjun.connector.doris.rest.DorisLoadClient.processGenericRowData(DorisLoadClient.java:170)
	at com.dtstack.chunjun.connector.doris.rest.DorisLoadClient.process(DorisLoadClient.java:156)
	at com.dtstack.chunjun.connector.doris.sink.DorisHttpOutputFormat.writeSingleRecordInternal(DorisHttpOutputFormat.java:89)
	... 22 more
@baisui1981 baisui1981 added bug Something isn't working 3.6.0-alpha labels Nov 12, 2022
@baisui1981 baisui1981 changed the title mysql -> doris 出错 mysql -> doris Flink 实时同步 出错 Nov 12, 2022
@baisui1981
Copy link
Member Author

已经支持 Flink SQL, 不会出错了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.6.0-alpha bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant