Skip to content

Commit

Permalink
make gtid enabled slave with less durable settings crash safe
Browse files Browse the repository at this point in the history
Summary:
With less durable settins (sync_binlog !=1 and
innodb_flush_log_at_trx_commit !=1), a slave with GTIDs/MTS
is not crash safe. Two possible scenarios causing slave
inconsistency

1) slave's binary log is behind innodb transaction log.
2) slave's binary log is ahead of innodb transaction log.

The slave_gtid_info transaction table consistently stores
gtid information and handles scenario 1. But in case
of scenario 2, even though the slave_gtid_info is consistent
with innodb, slave will skip executing some transactions if it's
GTID is logged in the binlog even though it is not committed in
innodb.

This diff fixes scenario 2 by changing gtid_executed when a
slave is initialized based on the binlog file and binlog position
which are logged inside innodb trx log. When gtid_executed is set
to an old value which is consistent with innodb, slave doesn't
miss any transactions.

Changing gtid_executed after crash recovery should be done only on slaves
with gtid enabled. Crash recovery on master should not do this because
it will create duplicate GTID log events for different transactions
causing data inconsistency on slaves.

Note that slaves may have duplicate GTID log events, but those
GTIDs corresponds to same transaction. Leaving the extra binlog
events on the slave doesn't create any problems because any slave
receiving those extra events skip executing any duplicate GTIDs.

Note that binlog file and binlog position in innodb header may be behind
binlog since there may be some transaction that are rolled
forward during crash recovery, but slave_gtid_info table handles
those cases by skipping the transactions but printing them to the binlog.

Test Plan:
repeatedly crashed a slave with innodb_flush_log_at_trx_commit=0

mtr_run --repeat=32 rpl_gtid_mts_stress_crash

Reviewers: rudradevbasak, jtolmer, yoshinori, tianx

Reviewed By: jtolmer, tianx
  • Loading branch information
santoshbanda authored and Herman Lee committed Jan 24, 2017
1 parent 55fd254 commit 00861dd
Show file tree
Hide file tree
Showing 10 changed files with 168 additions and 43 deletions.
3 changes: 2 additions & 1 deletion mysql-test/extra/rpl_tests/rpl_gtid_crash_safe.inc
Expand Up @@ -4,6 +4,7 @@ connection master;
-- let $uuid = `select @@server_uuid;`
create table t1(a int, PRIMARY KEY(a)) ENGINE=INNODB;
insert into t1 values(1);
insert into t1 values(2);
sync_slave_with_master;
use mysql;
-- replace_result $uuid uuid
Expand All @@ -14,7 +15,7 @@ connection slave;
-- exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.2.expect

connection master;
insert into t1 values(2);
insert into t1 values(3);

connection slave;
-- source include/wait_until_disconnected.inc
Expand Down
7 changes: 6 additions & 1 deletion mysql-test/extra/rpl_tests/rpl_parallel_load_innodb.test
Expand Up @@ -76,7 +76,7 @@ while($i)

# this table is special - just for timing. It's more special on test0 db
# where it contains master timing of the load as well.
create table benchmark (state text) engine=myisam; # timestamp keep on the slave side
create table benchmark (state text) engine=innodb; # timestamp keep on the slave side

dec $i;
}
Expand Down Expand Up @@ -258,6 +258,11 @@ connection slave;
--disable_result_log

insert into test0.benchmark set state='slave is processing load';
disable_query_log;
FLUSH LOGS;
enable_query_log;
let $wait_condition= SELECT count(*)+sleep(1) = 3 FROM test0.benchmark;
source include/wait_condition.inc;

# To force filling timestamp cols with the slave local clock values
# to implement benchmarking.
Expand Down

0 comments on commit 00861dd

Please sign in to comment.