Feature/partial update row event

[sean-k1]

Partial Update Row Event

Description

The partial update row event is an event introduced for performance in Mysql 8.0 JSON Update and is an inherited event of UpdateRowEvent.

The partial update row event is only heard for values that have changed for the Json column.
Therefore, it is not possible to determine if the value is actually null or has not changed.

To solve this issue, I think we need to further distinguish the Partial_Json_Null type like #489.
check rows["after_none_sources"]

*c:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedccc"]'}=>None(same with before values)
*d:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_updatedddd"])

when PartialUpdate events occured, we would be better give JsonDiff Object.

If you do not want after_values likes JsonDiff Obejct.
SET global binlog_row_value_options = ''; then you can get UpdateRowsEvents instead PartialUpdateRowsEvent

Example

sql query

SET  SESSION binlog_row_image = "FULL";
SET SESSION binlog_row_value_options = 'PARTIAL_JSON';
drop table if exists test_json_v2;
CREATE TABLE test_json_v2 (id INT, c JSON,PRIMARY KEY (id)) ;
ALTER TABLE test_json_v2 ADD COLUMN d JSON DEFAULT NULL, ADD COLUMN e JSON DEFAULT NULL;
INSERT INTO test_json_v2 VALUES (101,'{"a":"aaaaaaaaaaaaa", "c":"ccccccccccccccc", "ab":["abababababababa", "babababababab"]}', '{"a":"aaaaaaaaaaaaa", "c":"ccccccccccccccc", "ab":["abababababababa", "babababababab"]}','{"a":"aaaaaaaaaaaaa", "c":"ccccccccccccccc", "ab":["abababababababa", "babababababab"]}');
UPDATE test_json_v2 SET c = JSON_SET(c, '$.ab', '["ab_updatedccc"]') WHERE id = 101;
UPDATE test_json_v2 SET d = JSON_SET(d, '$.ab', '["ab_updatedddd"]') WHERE id = 101;
UPDATE test_json_v2 SET e = JSON_SET(e, '$.ab', '["ab_updatedeee"]'),c=NULL WHERE id = 101;
UPDATE test_json_v2 SET d = JSON_SET(d, '$.ab', '["ab_ddd"]'), e = json_set(e, '$.ab', '["ab_eee"]') WHERE id = 101;

Dump

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 421
Event size: 162
Read bytes: 162
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: DROP TABLE IF EXISTS `test_json_v2` /* generated by server */

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 685
Event size: 162
Read bytes: 162
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: CREATE TABLE test_json_v2 (id INT, c JSON,PRIMARY KEY (id))

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 977
Event size: 190
Read bytes: 190
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: ALTER TABLE test_json_v2 ADD COLUMN d JSON DEFAULT NULL, ADD COLUMN e JSON DEFAULT NULL

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 1150
Event size: 71
Read bytes: 71
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2023-10-14T15:05:02
Log position: 1250
Event size: 77
Read bytes: 77
Table id: 6488
Schema: pymysqlreplication_test
Table: test_json_v2
Columns: 4
=== OptionalMetaData ===
unsigned_column_list: [False, False, False, False]
default_charset_collation: None
charset_collation: {}
column_charset: []
column_name_list: ['id', 'c', 'd', 'e']
set_str_value_list : []
set_enum_str_value_list : []
geometry_type_list : []
simple_primary_key_list: [0]
primary_keys_with_prefix: {}
visibility_list: [True, True, True, True]
charset_collation_list: []
enum_and_set_collation_list: []

=== WriteRowsEvent ===
Date: 2023-10-14T15:05:02
Log position: 1602
Event size: 329
Read bytes: 12
Table: pymysqlreplication_test.test_json_v2
Affected columns: 4
Changed rows: 1
Column Name Information Flag: True
Values:
--
* id : 101
* c : {b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}
* d : {b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}
* e : {b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}

=== XidEvent ===
Date: 2023-10-14T15:05:02
Log position: 1633
Event size: 8
Read bytes: 8
Transaction ID: 33234

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 1815
Event size: 80
Read bytes: 80
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2023-10-14T15:05:02
Log position: 1915
Event size: 77
Read bytes: 77
Table id: 6488
Schema: pymysqlreplication_test
Table: test_json_v2
Columns: 4
=== OptionalMetaData ===
unsigned_column_list: [False, False, False, False]
default_charset_collation: None
charset_collation: {}
column_charset: []
column_name_list: ['id', 'c', 'd', 'e']
set_str_value_list : []
set_enum_str_value_list : []
geometry_type_list : []
simple_primary_key_list: [0]
primary_keys_with_prefix: {}
visibility_list: [True, True, True, True]
charset_collation_list: []
enum_and_set_collation_list: []

=== PartialUpdateRowsEvent ===
Date: 2023-10-14T15:05:02
Log position: 2313
Event size: 375
Read bytes: 13
Table: pymysqlreplication_test.test_json_v2
Affected columns: 4
Changed rows: 1
Column Name Information Flag: True
Values:
--
*id:101=>101
*c:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_updatedccc"])
*d:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>None(same with before values)
*e:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>None(same with before values)

=== XidEvent ===
Date: 2023-10-14T15:05:02
Log position: 2344
Event size: 8
Read bytes: 8
Transaction ID: 33235

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 2526
Event size: 80
Read bytes: 80
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2023-10-14T15:05:02
Log position: 2626
Event size: 77
Read bytes: 77
Table id: 6488
Schema: pymysqlreplication_test
Table: test_json_v2
Columns: 4
=== OptionalMetaData ===
unsigned_column_list: [False, False, False, False]
default_charset_collation: None
charset_collation: {}
column_charset: []
column_name_list: ['id', 'c', 'd', 'e']
set_str_value_list : []
set_enum_str_value_list : []
geometry_type_list : []
simple_primary_key_list: [0]
primary_keys_with_prefix: {}
visibility_list: [True, True, True, True]
charset_collation_list: []
enum_and_set_collation_list: []

=== PartialUpdateRowsEvent ===
Date: 2023-10-14T15:05:02
Log position: 3024
Event size: 375
Read bytes: 13
Table: pymysqlreplication_test.test_json_v2
Affected columns: 4
Changed rows: 1
Column Name Information Flag: True
Values:
--
*id:101=>101
*c:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedccc"]'}=>None(same with before values)
*d:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_updatedddd"])
*e:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>None(same with before values)

=== XidEvent ===
Date: 2023-10-14T15:05:02
Log position: 3055
Event size: 8
Read bytes: 8
Transaction ID: 33236

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 3237
Event size: 80
Read bytes: 80
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2023-10-14T15:05:02
Log position: 3337
Event size: 77
Read bytes: 77
Table id: 6488
Schema: pymysqlreplication_test
Table: test_json_v2
Columns: 4
=== OptionalMetaData ===
unsigned_column_list: [False, False, False, False]
default_charset_collation: None
charset_collation: {}
column_charset: []
column_name_list: ['id', 'c', 'd', 'e']
set_str_value_list : []
set_enum_str_value_list : []
geometry_type_list : []
simple_primary_key_list: [0]
primary_keys_with_prefix: {}
visibility_list: [True, True, True, True]
charset_collation_list: []
enum_and_set_collation_list: []

=== PartialUpdateRowsEvent ===
Date: 2023-10-14T15:05:02
Log position: 3731
Event size: 371
Read bytes: 13
Table: pymysqlreplication_test.test_json_v2
Affected columns: 4
Changed rows: 1
Column Name Information Flag: True
Values:
--
*id:101=>101
*c:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedccc"]'}=>None(null)
*d:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedddd"]'}=>None(same with before values)
*e:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': [b'abababababababa', b'babababababab']}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_updatedeee"])

=== XidEvent ===
Date: 2023-10-14T15:05:02
Log position: 3762
Event size: 8
Read bytes: 8
Transaction ID: 33237

=== QueryEvent ===
Date: 2023-10-14T15:05:02
Log position: 3944
Event size: 80
Read bytes: 80
Schema: b'pymysqlreplication_test'
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2023-10-14T15:05:02
Log position: 4044
Event size: 77
Read bytes: 77
Table id: 6488
Schema: pymysqlreplication_test
Table: test_json_v2
Columns: 4
=== OptionalMetaData ===
unsigned_column_list: [False, False, False, False]
default_charset_collation: None
charset_collation: {}
column_charset: []
column_name_list: ['id', 'c', 'd', 'e']
set_str_value_list : []
set_enum_str_value_list : []
geometry_type_list : []
simple_primary_key_list: [0]
primary_keys_with_prefix: {}
visibility_list: [True, True, True, True]
charset_collation_list: []
enum_and_set_collation_list: []

=== PartialUpdateRowsEvent ===
Date: 2023-10-14T15:05:02
Log position: 4346
Event size: 279
Read bytes: 13
Table: pymysqlreplication_test.test_json_v2
Affected columns: 4
Changed rows: 1
Column Name Information Flag: True
Values:
--
*id:101=>101
*c:None(null)=>None(null)
*d:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedddd"]'}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_ddd"])
*e:{b'a': b'aaaaaaaaaaaaa', b'c': b'ccccccccccccccc', b'ab': b'["ab_updatedeee"]'}=>JsonDiff(op :JsonDiffOperation.Replace path :$.ab value :["ab_eee"])

=== XidEvent ===
Date: 2023-10-14T15:05:02
Log position: 4377
Event size: 8
Read bytes: 8
Transaction ID: 33238

Ref

https://dev.mysql.com/blog-archive/partial-update-of-json-values/