Skip to content

Commit 555c12a

Browse files
bnesterebnestere
committed
MDEV-21087/MDEV-21433: ER_SLAVE_INCIDENT arrives at slave without failure specifics
Problem: ======= This patch addresses two issues: 1. An incident event can be incorrectly reported for transactions which are rolled back successfully. That is, an incident event should only be generated for failed “non-transactional transactions” (i.e., those which modify non-transactional tables) because they cannot be rolled back. 2. When the mariadb slave (error) stops at receiving the incident event there's no description of what led to it. Neither in the event nor in the master's error log. Solution: ======== Before reporting an incident event for a transaction, first validate that it is “non-transactional” (i.e. cannot be safely rolled back). To determine if a transaction is non-transactional, lex->stmt_accessed_table(LEX::STMT_WRITES_NON_TRANS_TABLE) is used because it is set previously in THD::decide_logging_format(). Additionally, when an incident event is written, write an error message to the server’s error log to indicate the underlying issue. Reviewed by: =========== Andrei Elkin <andrei.elkin@mariadb.com>
1 parent 46ff660 commit 555c12a

13 files changed

+778
-25
lines changed

‎mysql-test/suite/binlog/include/binlog_write_error.inc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ DROP TRIGGER IF EXISTS tr2;
3838
DROP VIEW IF EXISTS v1, v2;
3939
enable_warnings;
4040

41+
call mtr.add_suppression("Write to binary log failed: Error writing file*");
42+
4143
--echo #
4244
--echo # Test injecting binlog write error when executing queries
4345
--echo #

‎mysql-test/suite/binlog/r/binlog_bug23533.result

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size.*");
12
SET AUTOCOMMIT=0;
23
CREATE TABLE t1 (a INT NOT NULL AUTO_INCREMENT, b TEXT, PRIMARY KEY(a)) ENGINE=InnoDB;
34
SELECT COUNT(*) FROM t1;

‎mysql-test/suite/binlog/r/binlog_write_error.result

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ DROP PROCEDURE IF EXISTS p2;
99
DROP TRIGGER IF EXISTS tr1;
1010
DROP TRIGGER IF EXISTS tr2;
1111
DROP VIEW IF EXISTS v1, v2;
12+
call mtr.add_suppression("Write to binary log failed: Error writing file*");
1213
#
1314
# Test injecting binlog write error when executing queries
1415
#

‎mysql-test/suite/binlog/t/binlog_bug23533.test

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
--source include/have_innodb.inc
77
--source include/have_log_bin.inc
88
--source include/have_binlog_format_row.inc
9-
9+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size.*");
1010
SET AUTOCOMMIT=0;
1111

1212
# Create 1st table

‎mysql-test/suite/binlog_encryption/binlog_write_error.result

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ DROP PROCEDURE IF EXISTS p2;
99
DROP TRIGGER IF EXISTS tr1;
1010
DROP TRIGGER IF EXISTS tr2;
1111
DROP VIEW IF EXISTS v1, v2;
12+
call mtr.add_suppression("Write to binary log failed: Error writing file*");
1213
#
1314
# Test injecting binlog write error when executing queries
1415
#

‎mysql-test/suite/binlog_encryption/rpl_mixed_binlog_max_cache_size.result

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
include/master-slave.inc
22
[connection master]
33
call mtr.add_suppression("Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT");
4+
call mtr.add_suppression("Write to binary log failed: Multi-row statements required more than .max_binlog_stmt_cache_size");
5+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size");
46
SET GLOBAL max_binlog_cache_size = 4096;
57
SET GLOBAL binlog_cache_size = 4096;
68
SET GLOBAL max_binlog_stmt_cache_size = 4096;

‎mysql-test/suite/rpl/include/rpl_binlog_max_cache_size.test

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@
2222
#
2323
########################################################################################
2424
call mtr.add_suppression("Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT");
25+
call mtr.add_suppression("Write to binary log failed: Multi-row statements required more than .max_binlog_stmt_cache_size");
26+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size");
2527

2628
let $old_max_binlog_cache_size= query_get_value(SHOW VARIABLES LIKE "max_binlog_cache_size", Value, 1);
2729
let $old_binlog_cache_size= query_get_value(SHOW VARIABLES LIKE "binlog_cache_size", Value, 1);

‎mysql-test/suite/rpl/r/rpl_mdev-11092.result

Lines changed: 208 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
include/master-slave.inc
22
[connection master]
33
call mtr.add_suppression("Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT");
4-
call mtr.add_suppression("Slave SQL: The incident LOST_EVENTS occurred on the master. .*");
4+
call mtr.add_suppression("Slave SQL: The incident LOST_EVENTS occured on the master. .*");
5+
call mtr.add_suppression("Write to binary log failed: Multi-row statements required more than .max_binlog_stmt_cache_size.* ");
6+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size.* ");
7+
call mtr.add_suppression("Incident event write to the binary log file failed");
8+
call mtr.add_suppression("handlerton rollback failed");
9+
"*********** Annotate Event write failure **************"
510
SET GLOBAL max_binlog_cache_size = 4096;
611
SET GLOBAL binlog_cache_size = 4096;
712
SET GLOBAL max_binlog_stmt_cache_size = 4096;
@@ -10,12 +15,212 @@ disconnect master;
1015
connect master,127.0.0.1,root,,test,$MASTER_MYPORT,;
1116
CREATE TABLE t1(a INT PRIMARY KEY, data VARCHAR(30000)) ENGINE=MYISAM;
1217
connection master;
13-
ERROR HY000: Writing one row to the row-based binary log failed
18+
"#######################################################################"
19+
"# Test Case1: Annotate event write failure for MyISAM #"
20+
"#######################################################################"
21+
ERROR HY000: Multi-row statements required more than 'max_binlog_stmt_cache_size' bytes of storage; increase this mysqld variable and try again
22+
# Validating update was not binlogged..
23+
# ..success
24+
# Validating that the inserted data was saved on the master..
25+
# ..success
26+
connection slave;
1427
include/wait_for_slave_sql_error_and_skip.inc [errno=1590]
28+
# Validating that the insert was not replicated to the slave..
29+
# ..success
30+
"#######################################################################"
31+
"# Test Case2: Annotate event write failure for INNODB #"
32+
"#######################################################################"
1533
connection master;
34+
CREATE TABLE t2(a INT PRIMARY KEY, data VARCHAR(30000)) ENGINE=INNODB;
35+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
36+
# Validating binlog GTID position progressed from first insert..
37+
# ..success
38+
# Validating that only the first insert into t2 saved..
39+
# ..success
40+
include/save_master_gtid.inc
41+
connection slave;
42+
include/sync_with_master_gtid.inc
43+
# Validating the first insert into t2 replicated to slave..
44+
include/diff_tables.inc [master:test.t2,slave:test.t2]
45+
# ..success
46+
"#######################################################################"
47+
"# Test Case3: Annotate event write failure for mixed engine UPDATE #"
48+
"#######################################################################"
49+
connection master;
50+
ERROR HY000: Multi-row statements required more than 'max_binlog_stmt_cache_size' bytes of storage; increase this mysqld variable and try again
51+
# Validating update was not binlogged..
52+
# ..success
53+
# Validating non-transactional part of update saved..
54+
# ..success
55+
# Validating transactional part of update was rolled back..
56+
# ..success
57+
include/save_master_gtid.inc
58+
connection slave;
59+
include/wait_for_slave_sql_error_and_skip.inc [errno=1590]
60+
# Validating the rolled-back multi-engine update did not replicate to slave at all..
61+
# ..success
62+
connection master;
63+
"****** Clean up *******"
1664
SET GLOBAL max_binlog_cache_size= ORIGINAL_VALUE;
1765
SET GLOBAL binlog_cache_size= ORIGINAL_VALUE;
1866
SET GLOBAL max_binlog_stmt_cache_size= ORIGINAL_VALUE;
1967
SET GLOBAL binlog_stmt_cache_size= ORIGINAL_VALUE;
20-
DROP TABLE t1;
68+
DROP TABLE t1,t2;
69+
"*********** TABLE MAP Event write failure **************"
70+
CREATE TABLE tm (f INT) ENGINE=MYISAM;
71+
CREATE TABLE ti (f INT) ENGINE=INNODB;
72+
INSERT INTO tm VALUES (10);
73+
INSERT INTO ti VALUES (20);
74+
connection slave;
75+
"#######################################################################"
76+
"# Test Case4: Table_map event write failure for trans engine UPDATE #"
77+
"#######################################################################"
78+
# Transaction should be rolled back without writing incident event
79+
connection master;
80+
SET debug_dbug="+d,table_map_write_error";
81+
UPDATE ti, tm set ti.f=30;
82+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
83+
# Validating update was not binlogged..
84+
# ..success
85+
# Validating update was rolled back from storage engines..
86+
# ..success
87+
include/save_master_gtid.inc
88+
connection slave;
89+
include/sync_with_master_gtid.inc
90+
"#######################################################################"
91+
"# Test Case5: Table_map event write failure for mixed engine UPDATE #"
92+
"#######################################################################"
93+
connection master;
94+
# In case of mixed engines if non trans table is updated write INCIDENT event
95+
UPDATE ti,tm SET tm.f=88, ti.f=120;
96+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
97+
# Validating update was not binlogged..
98+
# ..success
99+
# Validating that only the non-transactional update saved on master..
100+
# ..success
101+
connection slave;
102+
include/wait_for_slave_sql_error_and_skip.inc [errno=1590]
103+
# Validating that neither of the updates replicated to slave..
104+
# ..success
105+
"#######################################################################"
106+
"# Test Case6: Committing a transaction consisting of two updates:
107+
"# S1) Update transactional table
108+
"# S2) Update transactional table
109+
"# with a table_map event write failure on the second event should
110+
"# roll-back only the second update without incident
111+
"#######################################################################"
112+
connection master;
113+
SET debug_dbug="";
114+
BEGIN;
115+
UPDATE ti, tm set ti.f=40;
116+
SET debug_dbug="+d,table_map_write_error";
117+
UPDATE ti, tm set ti.f=50;
118+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
119+
COMMIT;
120+
# Validating binlog GTID position progressed from first update..
121+
# ..success
122+
# Validating the first update saved..
123+
# ..and that the second update did not save..
124+
# ..success
125+
# Validating that only the first update replicated to slave without incident
126+
connection master;
127+
include/save_master_gtid.inc
128+
connection slave;
129+
include/sync_with_master_gtid.inc
130+
include/diff_tables.inc [master:test.ti,slave:test.ti]
131+
"#######################################################################"
132+
"# Test Case7: Rolling back a transaction consisting of two updates:
133+
"# S1) Update transactional table
134+
"# S2) Update transactional table
135+
"# with a table_map event write failure on the second event should
136+
"# roll-back both updates without incident
137+
"#######################################################################"
138+
connection master;
139+
SET debug_dbug="";
140+
BEGIN;
141+
UPDATE ti, tm set ti.f=60;
142+
SET debug_dbug="+d,table_map_write_error";
143+
UPDATE ti, tm set ti.f=70;
144+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
145+
ROLLBACK;
146+
# Validating update was not binlogged..
147+
# ..success
148+
# Validating that neither update saved on master..
149+
# ..success
150+
# Validating the transaction did not replicate to the slave
151+
connection master;
152+
include/save_master_gtid.inc
153+
connection slave;
154+
include/sync_with_master_gtid.inc
155+
include/diff_tables.inc [master:test.ti,slave:test.ti]
156+
"#######################################################################"
157+
"# Test Case8: Committing a transaction consisting of two updates:
158+
"# S1) Update transactional table
159+
"# S2) Update mixed trans/non-trans tables
160+
"# with a table_map event write failure on the second event should
161+
"# roll-back only the second update with incident
162+
"#######################################################################"
163+
connection master;
164+
BEGIN;
165+
SET debug_dbug="";
166+
UPDATE ti, tm set ti.f=80;
167+
SET debug_dbug="+d,table_map_write_error";
168+
UPDATE ti, tm set ti.f=90,tm.f=99;
169+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
170+
COMMIT;
171+
# Validating binlog GTID position progressed from first update..
172+
# ..success
173+
# Validating the first update saved..
174+
# ..and the transactional part of the second update did not save..
175+
# ..whereas the non-trans part of the second update did save..
176+
# ..success
177+
# Validating that the incident propagated to the slave
178+
connection slave;
179+
include/wait_for_slave_sql_error_and_skip.inc [errno=1590]
180+
# Validating that the first update replicated to the slave..
181+
# ..and neither part of the second update replicated..
182+
# ..success
183+
"#######################################################################"
184+
"# Test Case9: Rolling back a transaction consisting of two updates:
185+
"# S1) Update transactional table
186+
"# S2) Update mixed trans/non-trans tables
187+
"# with a table_map event write failure on the second event should
188+
"# roll-back both transactional updates, preserve the non-transactional
189+
"# update on the master (only), and write an incident event
190+
"#######################################################################"
191+
connection master;
192+
SET debug_dbug="";
193+
BEGIN;
194+
UPDATE ti, tm set ti.f=100;
195+
SET debug_dbug="+d,table_map_write_error";
196+
UPDATE ti, tm set ti.f=110,tm.f=111;
197+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
198+
ROLLBACK;
199+
# Validating update was not binlogged..
200+
# ..success
201+
# Validating trans updates rollback, but the non-trans update stays..
202+
# ..success
203+
# Validating that the incident propagated to the slave
204+
connection slave;
205+
include/wait_for_slave_sql_error_and_skip.inc [errno=1590]
206+
# Validating that none of the updates replicated to the slave
207+
include/diff_tables.inc [master:test.ti,slave:test.ti]
208+
# ..success
209+
"#######################################################################"
210+
"# Test Case10: If an incident event fails to write, a specific error
211+
"# should be logged
212+
"#
213+
"# Note: This test case is the same as test case 5, with the caveat of
214+
"# the incident event failing to write.
215+
"#######################################################################"
216+
connection master;
217+
SET debug_dbug="d,table_map_write_error,incident_event_write_error";
218+
UPDATE ti, tm set ti.f=120, tm.f=122;
219+
ERROR HY000: Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again
220+
# Validate error message indicating incident event failed to write
221+
FOUND 1 /Incident event write to the binary log file failed/ in mysqld.1.err
222+
connection master;
223+
"******** Clean Up **********"
224+
SET GLOBAL debug_dbug = '';
225+
DROP TABLE tm,ti;
21226
include/rpl_end.inc

‎mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
include/master-slave.inc
22
[connection master]
33
call mtr.add_suppression("Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT");
4+
call mtr.add_suppression("Write to binary log failed: Multi-row statements required more than .max_binlog_stmt_cache_size");
5+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size");
46
SET GLOBAL max_binlog_cache_size = 4096;
57
SET GLOBAL binlog_cache_size = 4096;
68
SET GLOBAL max_binlog_stmt_cache_size = 4096;

‎mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
include/master-slave.inc
22
[connection master]
33
call mtr.add_suppression("Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT");
4+
call mtr.add_suppression("Write to binary log failed: Multi-row statements required more than .max_binlog_stmt_cache_size");
5+
call mtr.add_suppression("Write to binary log failed: Multi-statement transaction required more than .max_binlog_cache_size");
46
SET GLOBAL max_binlog_cache_size = 4096;
57
SET GLOBAL binlog_cache_size = 4096;
68
SET GLOBAL max_binlog_stmt_cache_size = 4096;

0 commit comments

Comments
 (0)