xenon mysql rebuildme 报错

[zhangwx0794]

环境信息：
centos7.6
perconna mysql 5.7.39

xenon版本信息：
最新版
[mysql@Xenon-3 bin]$ ./xenoncli version
xenoncli:[{Tag: Time:2022/11/22 08:04:45 Git: GoVersion:go1.17.12 Platform:linux amd64}]

xenon mysql集群状态：

报错信息如下：

[mysql@Xenon-3 bin]$ /boss/soft/xenon/bin/xenoncli mysql backup --to=/u01/backup
2022/12/08 15:55:14.144046 [WARNING] rebuildme.found.best.slave[192.168.100.102:8801].leader[192.168.100.101:8801]
2022/12/08 15:55:14.144107 [WARNING] S1-->found.the.best.backup.host[192.168.100.102:8801]....
2022/12/08 15:55:14.148955 [WARNING] S2-->rm.and.mkdir.backupdir[/u01/backup]
2022/12/08 15:55:14.148971 [WARNING] S3-->xtrabackup.begin....
2022/12/08 15:55:14.149458 [WARNING] rebuildme.backup.req[&{From: BackupDir:/u01/backup SSHHost:192.168.100.103 SSHUser:mysql SSHPasswd:mysql SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[192.168.100.102:8801]
2022/12/08 15:55:33.293809 [WARNING] S3-->xtrabackup.end....
2022/12/08 15:55:33.293827 [WARNING] S4-->apply-log.begin....
2022/12/08 15:55:46.412498 [WARNING] S4-->apply-log.end....
2022/12/08 15:55:46.412509 [WARNING] completed OK!
2022/12/08 15:55:46.412512 [WARNING] backup.all.done....
[mysql@Xenon-3 bin]$ /boss/soft/xenon/bin/xenoncli mysql rebuildme --from=192.168.100.102:8801
2022/12/08 15:56:07.973953 [WARNING] =====prepare.to.rebuildme=====
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints "completed OK!".

2022/12/08 15:56:07.974236 [WARNING] S1-->check.raft.leader
2022/12/08 15:56:07.996902 [WARNING] S2-->prepare.rebuild.from[192.168.100.102:8801]....
2022/12/08 15:56:08.044707 [WARNING] S3-->check.bestone[192.168.100.102:8801].is.OK....
2022/12/08 15:56:08.044725 [WARNING] S4-->set.learner
2022/12/08 15:56:08.045802 [WARNING] S5-->stop.monitor
2022/12/08 15:56:08.046748 [WARNING] S6-->kill.mysql
2022/12/08 15:56:08.110453 [WARNING] S7-->check.bestone[192.168.100.102:8801].is.OK....
2022/12/08 15:56:08.227050 [WARNING] S8-->clear.datadir[/u01/backup]
2022/12/08 15:56:08.237190 [WARNING] S9-->xtrabackup.begin....
2022/12/08 15:56:08.237788 [WARNING] rebuildme.backup.req[&{From: BackupDir:/u01/backup SSHHost:192.168.100.103 SSHUser:mysql SSHPasswd:mysql SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[192.168.100.102:8801]
2022/12/08 15:56:22.364753 [WARNING] S9-->xtrabackup.end....
2022/12/08 15:56:22.364771 [WARNING] S10-->apply-log.begin....
2022/12/08 15:56:31.849770 [WARNING] S10-->apply-log.end....
2022/12/08 15:56:31.849780 [WARNING] S11-->start.mysql.begin...
2022/12/08 15:56:31.850831 [WARNING] S11-->start.mysql.end...
2022/12/08 15:56:31.850843 [WARNING] S12-->wait.mysqld.running.begin....
2022/12/08 15:56:34.878475 [WARNING] wait.mysqld.running...
2022/12/08 15:56:37.893701 [WARNING] wait.mysqld.running...
2022/12/08 15:56:37.906427 [WARNING] S12-->wait.mysqld.running.end....
2022/12/08 15:56:37.906441 [WARNING] S13-->wait.mysql.working.begin....
2022/12/08 15:56:40.917830 [WARNING] wait.mysql.working...
2022/12/08 15:56:40.918250 [WARNING] S13-->wait.mysql.working.end....
2022/12/08 15:56:40.918259 [WARNING] S14-->stop.and.reset.slave.begin....
2022/12/08 15:56:41.024755 [WARNING] S14-->stop.and.reset.slave.end....
2022/12/08 15:56:41.024768 [WARNING] S15-->reset.master.begin....
2022/12/08 15:56:41.028661 [WARNING] S15-->reset.master.end....
mysql.go:482: unexpected error: info.file.content.invalid[mysql_binlog.000002 3581
]

2022/12/08 15:56:41.036228 [PANIC] info.file.content.invalid[mysql_binlog.000002 3581
]
panic: [PANIC] info.file.content.invalid[mysql_binlog.000002 3581
]

goroutine 1 [running]:
xbase/xlog.(*Log).Panic(0xc0000e81d0, {0x7989c3, 0x7aa7b7}, {0xc0000d9ad0, 0xc0000d9ae0, 0x3})
/boss/soft/xenon/src/xbase/xlog/xlog.go:142 +0xc8
cli/cmd.ErrorOK({0x8130a0, 0xc000198650})
/boss/soft/xenon/src/cli/cmd/common.go:35 +0x1af
cli/cmd.mysqlRebuildMeCommandFn(0xc0000fbd40, {0xc0000e8260, 0x1, 0x1})
/boss/soft/xenon/src/cli/cmd/mysql.go:482 +0x15df
vendor/github.com/spf13/cobra.(*Command).execute(0xc0000fbd40, {0xc0000e8240, 0x1, 0x1})
/boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:603 +0x3a7
vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xa5c740)
/boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:689 +0x24b
vendor/github.com/spf13/cobra.(*Command).Execute(...)
/boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:648
main.main()
/boss/soft/xenon/src/cli/cli.go:43 +0x25

[caphash]

环境信息： centos7.6 perconna mysql 5.7.39

xenon版本信息： 最新版 [mysql@Xenon-3 bin]$ ./xenoncli version xenoncli:[{Tag: Time:2022/11/22 08:04:45 Git: GoVersion:go1.17.12 Platform:linux amd64}]

xenon mysql集群状态：

报错信息如下：

[mysql@Xenon-3 bin]$ /boss/soft/xenon/bin/xenoncli mysql backup --to=/u01/backup 2022/12/08 15:55:14.144046 [WARNING] rebuildme.found.best.slave[192.168.100.102:8801].leader[192.168.100.101:8801] 2022/12/08 15:55:14.144107 [WARNING] S1-->found.the.best.backup.host[192.168.100.102:8801].... 2022/12/08 15:55:14.148955 [WARNING] S2-->rm.and.mkdir.backupdir[/u01/backup] 2022/12/08 15:55:14.148971 [WARNING] S3-->xtrabackup.begin.... 2022/12/08 15:55:14.149458 [WARNING] rebuildme.backup.req[&{From: BackupDir:/u01/backup SSHHost:192.168.100.103 SSHUser:mysql SSHPasswd:mysql SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[192.168.100.102:8801] 2022/12/08 15:55:33.293809 [WARNING] S3-->xtrabackup.end.... 2022/12/08 15:55:33.293827 [WARNING] S4-->apply-log.begin.... 2022/12/08 15:55:46.412498 [WARNING] S4-->apply-log.end.... 2022/12/08 15:55:46.412509 [WARNING] completed OK! 2022/12/08 15:55:46.412512 [WARNING] backup.all.done.... [mysql@Xenon-3 bin]$ /boss/soft/xenon/bin/xenoncli mysql rebuildme --from=192.168.100.102:8801 2022/12/08 15:56:07.973953 [WARNING] =====prepare.to.rebuildme===== IMPORTANT: Please check that the backup run completes successfully. At the end of a successful backup run innobackupex prints "completed OK!".

2022/12/08 15:56:07.974236 [WARNING] S1-->check.raft.leader 2022/12/08 15:56:07.996902 [WARNING] S2-->prepare.rebuild.from[192.168.100.102:8801].... 2022/12/08 15:56:08.044707 [WARNING] S3-->check.bestone[192.168.100.102:8801].is.OK.... 2022/12/08 15:56:08.044725 [WARNING] S4-->set.learner 2022/12/08 15:56:08.045802 [WARNING] S5-->stop.monitor 2022/12/08 15:56:08.046748 [WARNING] S6-->kill.mysql 2022/12/08 15:56:08.110453 [WARNING] S7-->check.bestone[192.168.100.102:8801].is.OK.... 2022/12/08 15:56:08.227050 [WARNING] S8-->clear.datadir[/u01/backup] 2022/12/08 15:56:08.237190 [WARNING] S9-->xtrabackup.begin.... 2022/12/08 15:56:08.237788 [WARNING] rebuildme.backup.req[&{From: BackupDir:/u01/backup SSHHost:192.168.100.103 SSHUser:mysql SSHPasswd:mysql SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[192.168.100.102:8801] 2022/12/08 15:56:22.364753 [WARNING] S9-->xtrabackup.end.... 2022/12/08 15:56:22.364771 [WARNING] S10-->apply-log.begin.... 2022/12/08 15:56:31.849770 [WARNING] S10-->apply-log.end.... 2022/12/08 15:56:31.849780 [WARNING] S11-->start.mysql.begin... 2022/12/08 15:56:31.850831 [WARNING] S11-->start.mysql.end... 2022/12/08 15:56:31.850843 [WARNING] S12-->wait.mysqld.running.begin.... 2022/12/08 15:56:34.878475 [WARNING] wait.mysqld.running... 2022/12/08 15:56:37.893701 [WARNING] wait.mysqld.running... 2022/12/08 15:56:37.906427 [WARNING] S12-->wait.mysqld.running.end.... 2022/12/08 15:56:37.906441 [WARNING] S13-->wait.mysql.working.begin.... 2022/12/08 15:56:40.917830 [WARNING] wait.mysql.working... 2022/12/08 15:56:40.918250 [WARNING] S13-->wait.mysql.working.end.... 2022/12/08 15:56:40.918259 [WARNING] S14-->stop.and.reset.slave.begin.... 2022/12/08 15:56:41.024755 [WARNING] S14-->stop.and.reset.slave.end.... 2022/12/08 15:56:41.024768 [WARNING] S15-->reset.master.begin.... 2022/12/08 15:56:41.028661 [WARNING] S15-->reset.master.end.... mysql.go:482: unexpected error: info.file.content.invalid[mysql_binlog.000002 3581 ]

2022/12/08 15:56:41.036228 [PANIC] info.file.content.invalid[mysql_binlog.000002 3581 ] panic: [PANIC] info.file.content.invalid[mysql_binlog.000002 3581 ]

goroutine 1 [running]: xbase/xlog.(*Log).Panic(0xc0000e81d0, {0x7989c3, 0x7aa7b7}, {0xc0000d9ad0, 0xc0000d9ae0, 0x3}) /boss/soft/xenon/src/xbase/xlog/xlog.go:142 +0xc8 cli/cmd.ErrorOK({0x8130a0, 0xc000198650}) /boss/soft/xenon/src/cli/cmd/common.go:35 +0x1af cli/cmd.mysqlRebuildMeCommandFn(0xc0000fbd40, {0xc0000e8260, 0x1, 0x1}) /boss/soft/xenon/src/cli/cmd/mysql.go:482 +0x15df vendor/github.com/spf13/cobra.(*Command).execute(0xc0000fbd40, {0xc0000e8240, 0x1, 0x1}) /boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:603 +0x3a7 vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xa5c740) /boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:689 +0x24b vendor/github.com/spf13/cobra.(*Command).Execute(...) /boss/soft/xenon/src/vendor/github.com/spf13/cobra/command.go:648 main.main() /boss/soft/xenon/src/cli/cli.go:43 +0x25

应该是 MySQL 集群没有开启 GTID，所以rebuildme 时候获取不到 GTID 记录报错。
需要开启 GTID。