Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

解析binlog日志失败 #274

Closed
Christsam opened this issue Mar 3, 2017 · 3 comments
Closed

解析binlog日志失败 #274

Christsam opened this issue Mar 3, 2017 · 3 comments
Labels
Milestone

Comments

@Christsam
Copy link

Christsam commented Mar 3, 2017

canal运行后解析binlog日志的时候出现错误,提示Read Q_SQL_MODE_CODE error: limit excceed: 67
我看了canal的部分源码,貌似是sql_mode设置的不对?SQL_mode在canal里定义是64位的所以用getLong64,但是实际是67导致越界了?没看出数据库的数据有啥问题
PS: mysql版本是5.7.14

mysqlbinlog的日志:

见附件
binlog

binlog.txt

canal实例日志:

2017-03-03 17:17:19.952 [destination = gene_cachenotice_play3 , address = /172.16.10.213:3306 , EventParser] WARN c.a.otter.canal.parse.inbound.mysql.MysqlEventParser - prepare to find start position just last position
{"identity":{"slaveId":-1,"sourceAddress":{"address":"172.16.10.213","port":3306}},"postion":{"included":false,"journalName":"mysql-bin.000016","position":24856433,"serverId":213,"timestamp":1488484784000}}
2017-03-03 17:17:19.959 [destination = gene_cachenotice_play3 , address = /172.16.10.213:3306 , EventParser] WARN com.taobao.tddl.dbsync.binlog.LogDecoder - Decoding Query failed from: mysql-bin.000016:24858343
java.io.IOException: Read Q_SQL_MODE_CODE error: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:650) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.(QueryLogEvent.java:477) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:154) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:106) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:123) [canal.parse-1.0.23.jar:na]
at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:209) [canal.parse-1.0.23.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: java.lang.IllegalArgumentException: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.LogBuffer.getLong64(LogBuffer.java:873) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:566) ~[canal.parse.dbsync-1.0.23.jar:na]
... 6 common frames omitted
2017-03-03 17:17:19.960 [destination = gene_cachenotice_play3 , address = /172.16.10.213:3306 , EventParser] ERROR c.a.otter.canal.parse.inbound.mysql.MysqlEventParser - dump address 172.16.10.213/172.16.10.213:3306 has an error, retrying. caused by
java.io.IOException: Read Q_SQL_MODE_CODE error: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:650) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.(QueryLogEvent.java:477) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:154) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:106) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:123) ~[canal.parse-1.0.23.jar:na]
at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:209) ~[canal.parse-1.0.23.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: java.lang.IllegalArgumentException: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.LogBuffer.getLong64(LogBuffer.java:873) ~[canal.parse.dbsync-1.0.23.jar:na]
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:566) ~[canal.parse.dbsync-1.0.23.jar:na]
... 6 common frames omitted
2017-03-03 17:17:19.960 [destination = gene_cachenotice_play3 , address = /172.16.10.213:3306 , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:gene_cachenotice_play3[java.io.IOException: Read Q_SQL_MODE_CODE error: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:650)
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.(QueryLogEvent.java:477)
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:154)
at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:106)
at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:123)
at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:209)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: limit excceed: 67
at com.taobao.tddl.dbsync.binlog.LogBuffer.getLong64(LogBuffer.java:873)
at com.taobao.tddl.dbsync.binlog.event.QueryLogEvent.unpackVariables(QueryLogEvent.java:566)
... 6 more
]

@agapple
Copy link
Member

agapple commented Mar 17, 2017

mysql 5.7.14版本还没测试过,可能是有协议变更

agapple added a commit that referenced this issue Mar 17, 2017
@agapple
Copy link
Member

agapple commented Mar 17, 2017

改了一个代码,帮我验证一下mysql 5.7.14环境是否可以正常解析,谢谢。验证ok之后回复一下信息

@agapple agapple closed this as completed Apr 1, 2017
@agapple agapple added the bug label Apr 1, 2017
@agapple agapple added this to the v1.0.24 milestone Apr 1, 2017
@Jenvid
Copy link

Jenvid commented Jun 17, 2020

遇到同样的错
版本 5.7.16,noded 是1.0.20,尝试把 canal的 position 改到下一个位置,还是报同样的错

canal日志

2020-06-14 08:43:06.232 [destination = idc_canal_32xxTO32xxx , address = /192.168.32.xx:3306 , EventParser] WARN  com.taobao.tddl.dbsyn
c.binlog.LogDecoder - Decoding Query failed from: mysql-bin.003180:191252345
java.io.IOException: Read Q_SQL_MODE_CODE error: limit excceed: 75

binlog日志

### UPDATE `backupinfo`.`dba_database_states`
### WHERE
###   @1=1 /* TINYINT meta=0 nullable=0 is_null=0 */
###   @2='2020-06-14 08:42:02' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
### SET
###   @1=1 /* TINYINT meta=0 nullable=0 is_null=0 */
###   @2='2020-06-14 08:43:01' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
# at 191252249
#200614  8:43:01 server id 3278  end_log_pos 191252280 CRC32 0x5662ae39         Xid = 1278365063
COMMIT/*!*/;
# at 191252280
#200614  8:43:06 server id 3278  end_log_pos 191252345 CRC32 0x6784ca13         GTID    last_committed=139219   sequence_number=139220
SET @@SESSION.GTID_NEXT= '0ed25cd6-fed3-11e9-ae08-286ed588c673:418748102'/*!*/;
# at 191252345
#200614  8:43:00 server id 3278  end_log_pos 191252439 CRC32 0x58a6e853         Query   thread_id=47802988      exec_time=0     error_c
ode=0
SET TIMESTAMP=1592095380/*!*/;
SET @@session.sql_auto_is_null=1/*!*/;
SET @@session.explicit_defaults_for_timestamp=1/*!*/;
BEGIN
/*!*/;
# at 191252439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants