Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kafka 模式,控制台报canal 消费异常 #16

Closed
GITleonine1989 opened this issue Nov 7, 2023 · 4 comments
Closed

kafka 模式,控制台报canal 消费异常 #16

GITleonine1989 opened this issue Nov 7, 2023 · 4 comments

Comments

@GITleonine1989
Copy link

使用kafka 模式控制台报错canal 消费异常 回滚消息
2023-11-07 18:06:27.474 ERROR 33704 --- [l-client-thread] i.x.canal.core.client.KafkaCanalClient : canal 消费异常 回滚消息

com.alibaba.fastjson2.JSONException: error, offset 1, char �
at com.alibaba.fastjson2.JSONReaderUTF16.skipValue(JSONReaderUTF16.java:1585) ~[fastjson2-2.0.4.jar:na]
at com.alibaba.fastjson2.reader.ObjectReaderBean.processExtra(ObjectReaderBean.java:49) ~[fastjson2-2.0.4.jar:na]
at com.alibaba.fastjson2.reader.ObjectReaderBean.readObject(ObjectReaderBean.java:167) ~[fastjson2-2.0.4.jar:na]
at com.alibaba.fastjson2.JSON.parseObject(JSON.java:258) ~[fastjson2-2.0.4.jar:na]
at com.alibaba.otter.canal.client.kafka.KafkaCanalConnector.getFlatListWithoutAck(KafkaCanalConnector.java:233) ~[canal.client-1.1.6.jar:na]
at io.xzxj.canal.core.client.KafkaCanalClient.handleListening(KafkaCanalClient.java:31) ~[canal-client-core-0.0.7.jar:na]
at java.lang.Thread.run(Thread.java:750) ~[na:1.8.0_382]

打断点跟踪了一下,是在 getFlatListWithoutAck 这个方法里的 JSON.parseObject 报错,
我看 传给JSON.parseObject 好像是乱码

image

@GITleonine1989
Copy link
Author

乱码是因为 canal 的配置项 canal.mq.flatMessagep=false 导致的,改成true之后没有乱码 还是解析报错。。。

@xizixuejie
Copy link
Owner

可以贴一下kafka里的消息json,我试下嘛

@GITleonine1989
Copy link
Author

GITleonine1989 commented Nov 20, 2023

@xizixuejie 是canal ,那边的问题,升级到1.1.7 就好了,但是1.1.7他们引入的JSON包错了,导致了新的问题 alibaba/canal#4927
,我本地先把KafkaCanalConnector这个类的重写了一下(改成import fastjson2 ),解决了

@xizixuejie
Copy link
Owner

0.0.9版本改了,加上了MQConnector消费出现JSONException会忽略掉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants