Skip to content

Commit

Permalink
MDEV-32589 FULL_NODUP mode for binlog_row_image
Browse files Browse the repository at this point in the history
This patch provides a new mode FULL_NODUP to binlog_row_image system
variable. With FULL_NODUP mode, all columns are included in before
image, but only updated columns are included in after image for UPDATE.
While all columns are included in the after image for INSERT.

FULL_NODUP is for replacing FULL mode. It includes all data of
the before and after image as FULL mode, but it uses less storage
especially in the case that only a few columns are updated.

Note: It will binlog full before and after image for all modes if the
      table has no primary key. FULL_NODUP follows the behavior.
  • Loading branch information
SongLibing authored and LinuxJedi committed Nov 23, 2023
1 parent 6218b5f commit a119c5f
Show file tree
Hide file tree
Showing 20 changed files with 14,325 additions and 18 deletions.
20 changes: 11 additions & 9 deletions mysql-test/main/mysqld--help.result
Original file line number Diff line number Diff line change
Expand Up @@ -122,15 +122,17 @@ The following specify which files/extra groups are read (specified before remain
size if possible. The value has to be a multiple of 256.
--binlog-row-image=name
Controls whether rows should be logged in 'FULL',
'NOBLOB' or 'MINIMAL' formats. 'FULL', means that all
columns in the before and after image are logged.
'NOBLOB', means that mysqld avoids logging blob columns
whenever possible (eg, blob column was not changed or is
not part of primary key). 'MINIMAL', means that a PK
equivalent (PK columns or full row if there is no PK in
the table) is logged in the before image, and only
changed columns are logged in the after image. (Default:
FULL).
'FULL_NODUP', 'NOBLOB' or 'MINIMAL' formats. 'FULL',
means that all columns in the before and after image are
logged. 'FULL_NODUP', means that all columns are logged
in before image, but only changed columns or all columns
of inserted record are logged in after image, 'NOBLOB',
means that mysqld avoids logging blob columns whenever
possible (eg, blob column was not changed or is not part
of primary key). 'MINIMAL', means that a PK equivalent
(PK columns or full row if there is no PK in the table)
is logged in the before image, and only changed columns
are logged in the after image. (Default: FULL).
--binlog-row-metadata=name
Controls whether metadata is logged using FULL , MINIMAL
format and NO_LOG.FULL causes all metadata to be logged;
Expand Down
56 changes: 56 additions & 0 deletions mysql-test/suite/binlog/include/row_img.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Auxaliary script for test binlog_row_image
#
CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 varchar(100),
c3 INT DEFAULT 1000, c4 TEXT);
CREATE TABLE t2 (c1 INT, c2 char(100), c3 INT DEFAULT 1000, c4 TEXT);

FLUSH BINARY LOGS;
--let $binlog= query_get_value(SHOW MASTER STATUS, File, 1)
--let $datadir= `SELECT @@datadir`

--let $img_mode= `SELECT @@binlog_row_image`
--echo
--echo #########################################################################
--echo # binlog_row_image = $img_mode
--echo #########################################################################
--echo
INSERT INTO t1 VALUES(1, "insert_to_t1", 1, repeat('a', 20)),
(2, "insert_to_t1", 2, repeat('a', 20)),
(3, "insert_to_t1", 3, repeat('a', 20));
INSERT INTO t1(c1) VALUES(4);
UPDATE t1 SET c2 = "only_c2_changed";
UPDATE t1 SET c3 = 1, c4 = "c3_c4_changed";
DELETE FROM t1 WHERE c1 = 1;

--echo
--echo # Verify that rows events are binlogged as expeced.
--echo
--replace_result $MYSQLTEST_VARDIR MYSQLTEST_VARDIR
--replace_regex /^\# at .*// /.*SET (@@|TIMESTAMP).*// /.* end_log_pos .*// /xid=\d*/xid=<xid>/
--exec $MYSQL_BINLOG --force-if-open --verbose --verbose --base64-output=DECODE-ROWS $datadir/$binlog

FLUSH BINARY LOGS;
--let $binlog= query_get_value(SHOW MASTER STATUS, File, 1)
--let $datadir= `SELECT @@datadir`

--echo
--echo # t2 has no primary key.
--echo # It will binlog full before and after image for all modes if the
--echo # table has no primary key. FULL_NODUP follows the behavior.
--echo
INSERT INTO t2 VALUES(1, "insert_to_t2", 1, repeat('a', 20)),
(2, "insert_to_t2", 2, repeat('a', 20)),
(3, "insert_to_t2", 3, repeat('a', 20));
INSERT INTO t2(c1) VALUES(4);
UPDATE t2 SET c2 = "only_c2_changed";
UPDATE t2 SET c3 = 1, c4 = "c3_c4_changed";
DELETE FROM t2 WHERE c1 > 2;

--echo
--echo # Verify that rows events are binlogged as expeced.
--echo
--replace_result $MYSQLTEST_VARDIR MYSQLTEST_VARDIR
--replace_regex /^\# at .*// /.*SET (@@|TIMESTAMP).*// /.* end_log_pos .*// /xid=\d*/xid=<xid>/
--exec $MYSQL_BINLOG --force-if-open --verbose --verbose --base64-output=DECODE-ROWS $datadir/$binlog

DROP TABLE t1, t2;

0 comments on commit a119c5f

Please sign in to comment.