PS-8854: Convert binlog_utils_udf to a component with function auto-registration #10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion in mysqldump to disable"
PS-8581: Merge MySQL 8.0.32 - ASan: fix memory leak in group_replication.gr_add_node_with_ill_configurated_stack PS-8581: Merge MySQL 8.0.32 - Suppress Valgrind warnings in mach_read_from_1() called from rec_set_bit_field_1() PS-8581: Merge MySQL 8.0.32 - Valgrind: fix issue with ~vtoken_lock_cleanup PS-8581: Merge MySQL 8.0.32 - Valgrind: fix issue with net_read_packet() PS-8581: Merge MySQL 8.0.32 - Valgrind: fix issue with build_show_session_variables() PS-8581: Merge MySQL 8.0.32 - Valgrind: fix issue with os_fusionio_get_sector_size() PS-8581: Merge MySQL 8.0.32 - Valgrind: fix issue with Buf_fetch<Buf_fetch_normal>::single_page() and buf_page_init_low()
…routing_*" unit tests
…d cascade FKs Issue: When a multiple delete statement deletes both the parent and the child of a cascade foreign key, both deletes are written to the binary log. When the slave reads this log, it tries to delete the dependent row twice: first because the foreign key, and after because of the binlog entry. This bug was caused by WL#6987, dbb6e38 : before this refactoring, multi statement deletes executed deletes for one table immediately, then delayed the remaining entries. After the refactoring, foreign keys weren't handled. Fix: multi statement deletes fall back to deleyed mode when there is a cascade foreign key dependency between the tables.
…ys failing `audit_null.audit_plugin_bugs` doesn't work because ``` [ERROR] [MY-013129] [Server] A message intended for a client cannot be sent there as no client-session is attached. Therefore, we're sending the information to the error-log instead: MY-003164 - Abort message custom ``` doesn't match ``` let SEARCH_PATTERN= \[ERROR\] \[[^]]*\] \[[^]]*\] Abort message custom; ``` The real cause of wrong-formatting are changes in `auto_thd.cc` at mysql/mysql-server@c7154cf After fixes: ``` [ERROR] [MY-003164] [Server] Abort message custom ```
https://jira.percona.com/browse/PS-5966 Fixing by skipping already aggregated thread status variables when query for global status variables is invoked. Variables may be already aggregated, but thread can be still alive after aggregation in sql_class.cc THD::release_resources() PS-6052: gcc 9.2.1 reports new warnings PS-6054: clang 5.0.1 reports "missing-braces" warnings
https://jira.percona.com/browse/PS-5181 This is handling of corner case when: 1. gtid_next is set to GTID value 2. DROP statement with temporary and non-temporary tables is executed 3. row-based logging is enabled Theoretically such transaction should not be logged by master. It would be split into 2 separate transactions (SBR) or drop of temporary table would be skipped from logging, resulting in single table DROP for non-temporary table (RBR) However there are still at least 2 cases when it can happen: 1. Simulate replicated transaction by invoking DROP with 2 such tables directly on slave. 2. Create t1, t2 as non-temporary tables on master, then replicate to slave. Next drop t1, t2 from slave and create t1, t2 on slave only using stored procedure. This scenario is strange, however rpl_gtid_split_statements.test performs such scenario. If slave uses RBL, dropping of temporary table will not be logged, so single table drop of non-temporary table will go to slave's binlog. If slave uses SBL, case 1 above will issue the error (2 tables would be logged to binlog, but as separate transactions using the same GTID, which is not allowed). If slave uses SBL, case 2 will still work, as temporary tables created by stored procedures are not logged even if SBL is used.
mysqld --large-pages --innodb_buffer_pool_chunk_size=4G result in 1G chunk allocations instead of 4G. This patch was originally provided by Daniel Black at https://bugs.mysql.com/bug.php?id=94747 No testcase provided: the server setup requires a specific environment.
…ace_encrypt_X tests A situation may arise when a signal from (un)encryption process in background thread can arrive earlier than we start waiting on a conditional variable and mysql freezes
…h GROUP BY since 8.0.20) https://jira.percona.com/browse/PS-7019 From https://bugs.mysql.com/bug.php?id=99398: Refactoring work done in MySQL 8.0.20 caused single-row buffering for GROUP BY of non-nullable columns not to function correctly, not taking into account that such a column could be the inner table for an outer join, and thus would have a NULL flag that would need to be copied. In a GROUP BY without a temporary table, this would cause the NULL flag to come from the next output row instead of the previous one, and data returned to be inconsistent.
…ssion_value_ptr) https://jira.percona.com/browse/PS-7221 This is a regression introduced in 6d587a6. A new function called get_int_sort_key_for_item_inline temporarily sets the time_zone session variable to UTC. This results on *Time_zone_utc::get_name() been called and up to now, get_name was not meant to be called directly and thus always asserting. Adjusted the function to properly return UTC string instead of asserting.
…ostics_area::mysql_errno()` https://jira.percona.com/browse/PS-7231 Background ---------- In general, if a replication applier thread fails to execute a transaction because of an InnoDB deadlock or because the transaction's execution time exceeded InnoDB's innodb_lock_wait_timeout, it automatically retries slave_transaction_retries times before stopping with an error. And when --slave_preserve_commit_order is enabled, the replica server ensures that transactions are externalized on the replica in the same order as they appear in the replica's relay log, and prevents gaps in the sequence of transactions that have been executed from the relay log. If a thread's execution is completed before its preceding thread, then the executing thread waits until all previous transactions are committed before committing. Problem & Analysis ------------------ When --slave_preserve_commit_order is enabled on slave and if the waiting thread has locked the rows which are needed by the thread executing the previous transaction(as per relay log), then the innodb deadlock detection algorithm detects the deadlock between workers and will ask the waiting thread to rollback (only if its sequence number is lesser than that of the waiting thread). When this happens, the waiting thread wakes up from the cond_wait(SPCO) and it gets to know that it was asked to rollback by its preceding transaction as it was holding a lock that is needed by the other transaction to progress. It then rolls back its transaction so that the the preceding transaction can be committed and retries the transaction. But just before the transaction is retried, the worker checks if it encountered any errors during its execution. If there is no error, it simulates ER_LOCK_DEADLOCK error in order for it to be considered as a temporary error so that the worker thread retries the transaction. However, when the retries are exhausted, the worker thread logs an error into the error log by accessing the thread's diagnostic_area by calling `thd->get_stmt_da()->mysql_errno()`. If the error had been simulate (not called through `my_error` function call), the diagnostic_area would still be empty and thus making the assertion `DBUG_ASSERT(m_status == DA_ERROR);` to fail. NOTE: This assertion is observed only when both slave_transaction_retries and slave_preserve_commit_order are enabled on the replica server and is more likely to happen when slave_transaction_retries is set to a lower value. Fix --- `Slave_transaction:::retry_transaction()` has been modified to call `thd->get_stmt_da()->mysql_errno()` only when `thd->is_error()` is evaluated to true.
…il_system->shard_by_id(id)->mutex_owned() (percona#4364) https://jira.percona.com/browse/PS-7666 Problem: -------- On debug builds, a crash during DROP TABLE operation, makes the database unusable. InnoDB will enter crash recovery but it will keep crashing with the same assertion failure For the DROP TABLE operation, InnoDB writes MLOG_FILE_DELETE redo log entry. A subsequent start, InnoDB will enter crash recovery and tries to replay the operation. During this replay, it tries to delete the file again and remove the in-memory tablespace object (fil_space_t). Removal of the in-memory tablespace object fails with assertion failure as it expect the Fil_system->Fil_shard mutex to be taken. Fix: ---- Acquire Fil_system->Fil_shard mutex for the in-memory space removal and release after it is done with removal
…dated in partitioned table https://jira.percona.com/browse/PS-7856 Use dbug_tmp_use_all_columns/dbug_tmp_restore_column_map as described in 74cc73d as a workaround, because autoincremented column is not used in the query.
…ALTER TABLESPACE https://jira.percona.com/browse/PS-7923 Problem: Metadata locks upgraded during ALTER TABLESPACE are upgraded in no particular order, which present deadlock possibility. Metadata deadlock prevention mechanism kicks in and rolls back an attachable transaction during data dictionary updates, which causes the server to terminate. Fix: Sort metadata lock requests before upgrade procedure. Move metadata lock upgrading routine to MDL_context class to reuse the sorting comparator.
…mn crashes in innobase_get_computed_value https://jira.percona.com/browse/PS-7940 Problem: Virtual column template of child table is not populated when cascade delete happened on parent table. Failing to initliaze it causes a crash when we are trying remove the secondary index rows defined on virtual column . Analysis: A cascade delete from the parent table triggers delete on the child table. Before a clustered index record is deleted in the child table, a copy of row is built to remove secondary index records. This copy of the row requires virtual columns to be materialized. Hence, if child table has any virtual columns, we have to initialize virtual column template Fix: As part of fix e5e8c9d MySQL handle case when child virtual column depends on forign key. To extend the fix we are now initliazing virtual column template if child table has any virtual column. (cherry picked from commit 4bdd2da)
…n while it is replicating a CTAS. https://jira.percona.com/browse/PS-8106 On rollback of CREATE TABLE .. AS SELECT statement DDL Log REMOVE CACHE is executed, but underlying procedure dict_table_remove_from_cache_low requires that table shouldn't have opened handles (get_ref_count() == 0). If tables were closed before rollback, this procedure can be executed freely.
…t from performance_schema is broken https://jira.percona.com/browse/PS-8128 View is broken because when it created, perfromance_schema wasn't imported yet. Here we swap code populating DD tables and code initializing performance_schema tables. This allows to upgrade views that refer to performance_schema tables.
…UT_LIST_GET_LEN(buf_pool->flush_list) == 0 https://jira.percona.com/browse/PS-8174 Problem: There is a possibility that at shutdown by the time we do the last sweep on flushing the buffer pool there are still pages in the flush list. Those pages are still marked as io_fix->BUF_IO_READ thus they are not eligible for flushing from flush_list. Where is the workflow: 1. ibuf_merge_in_background requested those pages to be read in order to merge the ibuf changes. This will mark the page as BUF_IO_READ and increment buf_pool->n_pend_reads by 1. 2. When IO threads pick them up, it will start to merge the insert bugger changes. 3. On the first change, it will add the page to flush_list. 4. If there are more changes to apply, it will and continue on applying the changes until it is done. 5. Once the io thread finishes applying ibuf records to this page, it will mark the page as BUF_IO_NONE 6. the io thread decreases buf_pool->n_pend_reads by 1. The last sweep on flushing buffer pool considers the round of flushes completed when n_flushed == 0 which is not correct, if it runs when we are at step 4. Also, there is a still another race condition that by the time we tried to flush a page it was still marked as BUF_IO_READ (step 5), but when the page cleaner code checks buf_get_n_pending_read_ios, the IO thread has already decremented leaving one last page on flush_list. Fix: For the last round flushing pages at shutdown, only consider the flush completed if there are no pending read io operations and flush_list size is 0. Added a new function to get flush_list lenght. For this last sweep from PC we do not need mutex as there won't be other threads reading nor writting to it at this stage.
… to allow user to specify a consumer name) https://jira.percona.com/browse/PS-8385 Problem: Currently implementation of redo log consumer UDF does not allow users to specify a custom name for the consumer. This can be misleading when ER_IB_MSG_LOG_WRITER_WAIT_ON_CONSUMER error is issued as it will always report MEB as consumer name. Fix: Adjust the UDF to allow users to optionally specify a consumer name. In case of no consumer name specified, assume MEB. Also adjusted Log_consumer class to have a consumer_type ENUM. This is used log_writer_wait_on_consumers to validate we are not waiting on CONSUMER_TYPE_SERVER (log archive or checkpointer).
…fe.sh MySQL 8.0 has added handling of MYSQLD_RESTART_EXIT (16) which conflicts with handling of mysqld's return values ported from PS-5.7. This patch fixes `main.restart_server` MTR test.
…r generated and upgraded sys tables) https://jira.percona.com/browse/PS-7811 Test main.upgrade_system_tables complained that explicit_encryption system table option is different for freshly generated system tables and ones upgraded from 5.7. The reason for this is how new table creation in mysql_system_tables.sql and table upgrade in mysql_system_tables_fix.sql is done. In mysql_system_tables.sql there is something similar to SET @cmd = "CREATE TABLE IF NOT EXISTS columns_priv ( ... )"; SET @str = CONCAT(@cmd, " ENCRYPTION='", @is_mysql_encrypted, "'"); Here ENCRYPTION for a table is set explicitly not depending if it is ON or OFF. As a result we always have explicit_encryption=1 for generated table and this explicit settings will be used to decide if table should be encrypted or not. At the same time in mysql_system_tables_fix.sql we have SET @str="ALTER TABLE mysql.columns_priv ENCRYPTION='Y'"; SET @cmd = IF(STRCMP(@is_mysql_encrypted,'Y'), 'SET @dummy = 0', @str); So this ALTER isn't performed in case encryption is disabled and we get explicit_encryption=0 option for upgraded table. This will lead to actual table encryption setting being taken from global defaults. As a result we get different behaviour in relation to table encryption setting for newly generated and upgraded tables. To fix this inconsistency the mysql_system_tables_fix.sql was updated to always run mentioned ALTER with explicit ENCRYPTION settings and as a result setting explicit_encryption=1 for upgraded tables.
…h ps-admin) https://jira.percona.com/browse/PS-7917 Problem New tokudb deprecation error was logging a message before logging service for plugins was initialized. Fix Moved the deprecation error to after logging service initialization. Adjusted error message to display loose-X configuration. Otherwise, server will fail to start if tokudb_enabled or tokudb_backup_enabled are specified but plugin is not loaded. Adjusted error message to not mention that those variables require a server restart. They will be read at plugin initialization, thus not requiring a complete server restart. Added check at ps-admin to also display the deprecation error message.
https://jira.percona.com/browse/PS-8848 The Connection Control plugin is designed to introduce random delay for a user connection attempt in case number of failing attempts reaches some threshold. At the moment plugin adds this delay silently so you cannot notice the plugin is actually doing something by looking into server's logs. Improved plugin to write an information message to server's error log in case number of failed connection attempts crosses the threshold. To control number of those messages printed to log in cases of a big number of failing connection attempts for different accounts, the limit of 10 messages per minute is hardcoded. In case messages for some connection attempts were suppressed during the interval, there will be a message reporting overall number of delayed connection attempts during this period in the end of each interval.
1. Use `$REAL_COMPILER_VER` instead of `$(CompilerVer)`. 2. The method using `bc <<< "$(CompilerVer)` works only with 2 digits (e.g. `7.4`). It doesn't work with 3+ digits (e.g. `7.4.1`).
…am_hypergraph, index_merge_innodb, index_merge_rocksdb2
…omponent with function auto-registration
percona-ysorokin
suggested changes
Aug 21, 2023
There was a problem hiding this comment.
Please also wait for the Masking Functions component PR - it has function auto-registration code put in common headers.
In addition, the new UDF wrappers framework has new extensions for working with character sets, so it would be really nice if the UDFs that return STRING here would also explicitly set charset to ascii for the return value (by default it is binary that with default mysql client options is shown as hexadecimal value)
| MODULE_ONLY | ||
| ) | ||
|
|
||
| ADD_DEFINITIONS(-DMYSQL_SERVER) |
There was a problem hiding this comment.
Better use
target_compile_definitions(component_binlog_utils_udf PRIVATE MYSQL_SERVER)
Also, check if this MYSQL_SERVER is still needed (with the binlog stream classes refactoring Oracle did in 8.0.34, it may no longer be needed)
inikep
force-pushed
the
internal-8.1.0-linear
branch
from
84d6196
to
ecdcdf4
Compare
inikep
added a commit
that referenced
this pull request
Aug 24, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
inikep
force-pushed
the
internal-8.1.0-linear
branch
5 times, most recently
from
10eb6c3
to
cd912d1
Compare
inikep
added a commit
that referenced
this pull request
Oct 16, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Oct 16, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Oct 25, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Oct 26, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Oct 26, 2023
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Jan 17, 2024
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
added a commit
that referenced
this pull request
Jan 17, 2024
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
pushed a commit
that referenced
this pull request
Feb 8, 2024
When a node starts up, it might connect to a management server to obtain the cluster configuration. The component that does this is known as ConfigRetriever. This patch adds an instance of TlsKeyManager to ConfigRetriever for use in MGM TLS by both sorts of ndbd processes: the data node and the angel process. The data node (but not the angel) also has a TlsKeyManager in the global TransporterRegistry. In the data node, these two TlsKeyManagers are initialized with the same node type and TLS search path, so they will both use the same certificate. In a data node, ConfigRetriever's MGM connection is the one that will get turned in to a transporter. In the existing MTR test tls_off_certs, the behavior changes so that the data nodes now have TLS connections to the MGM node. An NDB management server might also be a client of some other NDB management server. This patch adds support for the --ndb-mgm-tls option to ndb_mgmd, for setting its MGM TLS requirement level when it acts as a client. This patch extends the test testMgmd -n NdbdWithCertificate to run with [MGM]RequireTls set to true. Change-Id: I9ab68bef61b80a18edbf1375366c81ed874d7026
inikep
added a commit
that referenced
this pull request
Feb 8, 2024
PS-5741: Incorrect use of memset_s in keyring_vault.
Fixed the usage of memset_s. The arguments should be:
void memset_s(void *dest, size_t dest_max, int c, size_t n)
where the 2nd argument is size of buffer and the 3rd is
argument is character to fill.
---------------------------------------------------------------------------
PS-7769 - Fix use-after-return error in audit_log_exclude_accounts_validate
---
*Problem:*
`st_mysql_value::val_str` might return a pointer to `buf` which after
the function called is deleted. Therefore the value in `save`, after
reuturnin from the function, is invalid.
In this particular case, the error is not manifesting as val_str`
returns memory allocated with `thd_strmake` and it does not use `buf`.
*Solution:*
Allocate memory with `thd_strmake` so the memory in `save` is not local.
---------------------------------------------------------------------------
Fix test main.bug12969156 when WITH_ASAN=ON
*Problem:*
ASAN complains about stack-buffer-overflow on function `mysql_heartbeat`:
```
==90890==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fe746d06d14 at pc 0x7fe760f5b017 bp 0x7fe746d06cd0 sp 0x7fe746d06478
WRITE of size 24 at 0x7fe746d06d14 thread T16777215
Address 0x7fe746d06d14 is located in stack of thread T26 at offset 340 in frame
#0 0x7fe746d0a55c in mysql_heartbeat(void*) /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:62
This frame has 4 object(s):
[48, 56) 'result' (line 66)
[80, 112) '_db_stack_frame_' (line 63)
[144, 200) 'tm_tmp' (line 67)
[240, 340) 'buffer' (line 65) <== Memory access at offset 340 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
Thread T26 created by T25 here:
#0 0x7fe760f5f6d5 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x557ccbbcb857 in my_thread_create /home/yura/ws/percona-server/mysys/my_thread.c:104
#2 0x7fe746d0b21a in daemon_example_plugin_init /home/yura/ws/percona-server/plugin/daemon_example/daemon_example.cc:148
#3 0x557ccb4c69c7 in plugin_initialize /home/yura/ws/percona-server/sql/sql_plugin.cc:1279
#4 0x557ccb4d19cd in mysql_install_plugin /home/yura/ws/percona-server/sql/sql_plugin.cc:2279
#5 0x557ccb4d218f in Sql_cmd_install_plugin::execute(THD*) /home/yura/ws/percona-server/sql/sql_plugin.cc:4664
#6 0x557ccb47695e in mysql_execute_command(THD*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5160
#7 0x557ccb47977c in mysql_parse(THD*, Parser_state*, bool) /home/yura/ws/percona-server/sql/sql_parse.cc:5952
#8 0x557ccb47b6c2 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/yura/ws/percona-server/sql/sql_parse.cc:1544
#9 0x557ccb47de1d in do_command(THD*) /home/yura/ws/percona-server/sql/sql_parse.cc:1065
#10 0x557ccb6ac294 in handle_connection /home/yura/ws/percona-server/sql/conn_handler/connection_handler_per_thread.cc:325
#11 0x557ccbbfabb0 in pfs_spawn_thread /home/yura/ws/percona-server/storage/perfschema/pfs.cc:2198
#12 0x7fe760ab544f in start_thread nptl/pthread_create.c:473
```
The reason is that `my_thread_cancel` is used to finish the daemon thread. This is not and orderly way of finishing the thread. ASAN does not register the stack variables are not used anymore which generates the error above.
This is a benign error as all the variables are on the stack.
*Solution*:
Finish the thread in orderly way by using a signalling variable.
---------------------------------------------------------------------------
PS-8204: Fix XML escape rules for audit plugin
https://jira.percona.com/browse/PS-8204
There was a wrong length specified for some XML
escape rules. As a result of this terminating null symbol from
replacement rule was copied into resulting string. This lead to
quer text truncation in audit log file.
In addition added empty replacement rules for '\b' and 'f' symbols
which just remove them from resulting string. These symboles are
not supported in XML 1.0.
---------------------------------------------------------------------------
PS-8854: Add main.percona_udf MTR test
Add a test to check FNV1A_64, FNV_64, and MURMUR_HASH user-defined functions.
inikep
pushed a commit
that referenced
this pull request
Mar 15, 2024
…ocal DDL
executed
https://perconadev.atlassian.net/browse/PS-9018
Problem
-------
In high concurrency scenarios, MySQL replica can enter into a deadlock due to a
race condition between the replica applier thread and the client thread
performing a binlog group commit.
Analysis
--------
It needs at least 3 threads for this deadlock to happen
1. One client thread
2. Two replica applier threads
How this deadlock happens?
--------------------------
0. Binlog is enabled on replica, but log_replica_updates is disabled.
1. Initially, both "Commit Order" and "Binlog Flush" queues are empty.
2. Replica applier thread 1 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
3. Since both "Commit Order" and "Binlog Flush" queues are empty, the applier
thread 1
3.1. Becomes leader (In Commit_stage_manager::enroll_for()).
3.2. Registers in the commit order queue.
3.3. Acquires the lock MYSQL_BIN_LOG::LOCK_log.
3.4. Commit Order queue is emptied, but the lock MYSQL_BIN_LOG::LOCK_log is
not yet released.
NOTE: SE commit for applier thread is already done by the time it reaches
here.
4. Replica applier thread 2 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
5. Since the "Commit Order" queue is empty (emptied by applier thread 1 in 3.4), the
applier thread 2
5.1. Becomes leader (In Commit_stage_manager::enroll_for())
5.2. Registers in the commit order queue.
5.3. Tries to acquire the lock MYSQL_BIN_LOG::LOCK_log. Since it is held by applier
thread 1 it will wait until the lock is released.
6. Client thread enters the group commit pipeline to register in the
"Binlog Flush" queue.
7. Since "Commit Order" queue is not empty (there is applier thread 2 in the
queue), it enters the conditional wait `m_stage_cond_leader` with an
intention to become the leader for both the "Binlog Flush" and
"Commit Order" queues.
8. Applier thread 1 releases the lock MYSQL_BIN_LOG::LOCK_log and proceeds to update
the GTID by calling gtid_state->update_commit_group() from
Commit_order_manager::flush_engine_and_signal_threads().
9. Applier thread 2 acquires the lock MYSQL_BIN_LOG::LOCK_log.
9.1. It checks if there is any thread waiting in the "Binlog Flush" queue
to become the leader. Here it finds the client thread waiting to be
the leader.
9.2. It releases the lock MYSQL_BIN_LOG::LOCK_log and signals on the
cond_var `m_stage_cond_leader` and enters a conditional wait until the
thread's `tx_commit_pending` is set to false by the client thread
(will be done in the
Commit_stage_manager::process_final_stage_for_ordered_commit_group()
called by client thread from fetch_and_process_flush_stage_queue()).
10. The client thread wakes up from the cond_var `m_stage_cond_leader`. The
thread has now become a leader and it is its responsibility to update GTID
of applier thread 2.
10.1. It acquires the lock MYSQL_BIN_LOG::LOCK_log.
10.2. Returns from `enroll_for()` and proceeds to process the
"Commit Order" and "Binlog Flush" queues.
10.3. Fetches the "Commit Order" and "Binlog Flush" queues.
10.4. Performs the storage engine flush by calling ha_flush_logs() from
fetch_and_process_flush_stage_queue().
10.5. Proceeds to update the GTID of threads in "Commit Order" queue by
calling gtid_state->update_commit_group() from
Commit_stage_manager::process_final_stage_for_ordered_commit_group().
11. At this point, we will have
- Client thread performing GTID update on behalf if applier thread 2 (from step 10.5), and
- Applier thread 1 performing GTID update for itself (from step 8).
Due to the lack of proper synchronization between the above two threads,
there exists a time window where both threads can call
gtid_state->update_commit_group() concurrently.
In subsequent steps, both threads simultaneously try to modify the contents
of the array `commit_group_sidnos` which is used to track the lock status of
sidnos. This concurrent access to `update_commit_group()` can cause a
lock-leak resulting in one thread acquiring the sidno lock and not
releasing at all.
-----------------------------------------------------------------------------------------------------------
Client thread Applier Thread 1
-----------------------------------------------------------------------------------------------------------
update_commit_group() => global_sid_lock->rdlock(); update_commit_group() => global_sid_lock->rdlock();
calls update_gtids_impl_lock_sidnos() calls update_gtids_impl_lock_sidnos()
set commit_group_sidno[2] = true set commit_group_sidno[2] = true
lock_sidno(2) -> successful
lock_sidno(2) -> waits
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) {
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Applier thread continues..
lock_sidno(2) -> successful
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) { <=== this check fails and lock is not released.
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Client thread continues without releasing the lock
-----------------------------------------------------------------------------------------------------------
12. As the above lock-leak can also happen the other way i.e, the applier
thread fails to unlock, there can be different consequences hereafter.
13. If the client thread continues without releasing the lock, then at a later
stage, it can enter into a deadlock with the applier thread performing a
GTID update with stack trace.
Client_thread
-------------
#1 __GI___lll_lock_wait
#2 ___pthread_mutex_lock
#3 native_mutex_lock <= waits for commit lock while holding sidno lock
#4 Commit_stage_manager::enroll_for
#5 MYSQL_BIN_LOG::change_stage
#6 MYSQL_BIN_LOG::ordered_commit
#7 MYSQL_BIN_LOG::commit
#8 ha_commit_trans
#9 trans_commit_implicit
#10 mysql_create_like_table
#11 Sql_cmd_create_table::execute
#12 mysql_execute_command
percona#13 dispatch_sql_command
Applier thread
--------------
#1 ___pthread_mutex_lock
#2 native_mutex_lock
#3 safe_mutex_lock
#4 Gtid_state::update_gtids_impl_lock_sidnos <= waits for sidno lock
#5 Gtid_state::update_commit_group
#6 Commit_order_manager::flush_engine_and_signal_threads <= acquires commit lock here
#7 Commit_order_manager::finish
#8 Commit_order_manager::wait_and_finish
#9 ha_commit_low
#10 trx_coordinator::commit_in_engines
#11 MYSQL_BIN_LOG::commit
#12 ha_commit_trans
percona#13 trans_commit
percona#14 Xid_log_event::do_commit
percona#15 Xid_apply_log_event::do_apply_event_worker
percona#16 Slave_worker::slave_worker_exec_event
percona#17 slave_worker_exec_job_group
percona#18 handle_slave_worker
14. If the applier thread continues without releasing the lock, then at a later
stage, it can perform recursive locking while setting the GTID for the next
transaction (in set_gtid_next()).
In debug builds the above case hits the assertion
`safe_mutex_assert_not_owner()` meaning the lock is already acquired by the
replica applier thread when it tries to re-acquire the lock.
Solution
--------
In the above problematic example, when seen from each thread
individually, we can conclude that there is no problem in the order of lock
acquisition, thus there is no need to change the lock order.
However, the root cause for this problem is that multiple threads can
concurrently access to the array `Gtid_state::commit_group_sidnos`.
In its initial implementation, it was expected that threads should
hold the `MYSQL_BIN_LOG::LOCK_commit` before modifying its contents. But it
was not considered when upstream implemented WL#7846 (MTS:
slave-preserve-commit-order when log-slave-updates/binlog is disabled).
With this patch, we now ensure that `MYSQL_BIN_LOG::LOCK_commit` is acquired
when the client thread (binlog flush leader) when it tries to perform GTID
update on behalf of threads waiting in "Commit Order" queue, thus providing a
guarantee that `Gtid_state::commit_group_sidnos` array is never accessed
without the protection of `MYSQL_BIN_LOG::LOCK_commit`.
inikep
pushed a commit
that referenced
this pull request
Apr 9, 2024
inikep
pushed a commit
that referenced
this pull request
Apr 9, 2024
Summary: Rebaseline type relate tests: 1. SHOW INDEX format changes 2. No more LSM_TREE in Index_type as the corresponding 3. More rows in EXPLAIN and rows column is now in column #10 (instead of #9) and there is another filtered column that needs to be masked 4. Using utf8 results warnings but I decided to keep the warning as a reminder in the future. TODO: we need to decide what to do with binary unpacking with regarding to utf8/utf8mb3/utf8mb4. 5. YEAR(2) is deprecated 6. GROUP BY now needs ORDER BY for ordered output 7. MySQL 8 no longer allows specifying NULL for PK so I'm adding a extra_pk_col_opts to the col_opt_*.tests to only specify supported flags for PK 8. Binary output are now hex instead of character (for example, 0 => 0x30) Reviewed By: lloyd Differential Revision: D17503619
inikep
pushed a commit
that referenced
this pull request
Jun 14, 2024
…ocal DDL
executed
https://perconadev.atlassian.net/browse/PS-9018
Merge remote-tracking branch 'venki/PS-9018-8.0-gca' into HEAD
Problem
-------
In high concurrency scenarios, MySQL replica can enter into a deadlock due to a
race condition between the replica applier thread and the client thread
performing a binlog group commit.
Analysis
--------
It needs at least 3 threads for this deadlock to happen
1. One client thread
2. Two replica applier threads
How this deadlock happens?
--------------------------
0. Binlog is enabled on replica, but log_replica_updates is disabled.
1. Initially, both "Commit Order" and "Binlog Flush" queues are empty.
2. Replica applier thread 1 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
3. Since both "Commit Order" and "Binlog Flush" queues are empty, the applier
thread 1
3.1. Becomes leader (In Commit_stage_manager::enroll_for()).
3.2. Registers in the commit order queue.
3.3. Acquires the lock MYSQL_BIN_LOG::LOCK_log.
3.4. Commit Order queue is emptied, but the lock MYSQL_BIN_LOG::LOCK_log is
not yet released.
NOTE: SE commit for applier thread is already done by the time it reaches
here.
4. Replica applier thread 2 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
5. Since the "Commit Order" queue is empty (emptied by applier thread 1 in 3.4), the
applier thread 2
5.1. Becomes leader (In Commit_stage_manager::enroll_for())
5.2. Registers in the commit order queue.
5.3. Tries to acquire the lock MYSQL_BIN_LOG::LOCK_log. Since it is held by applier
thread 1 it will wait until the lock is released.
6. Client thread enters the group commit pipeline to register in the
"Binlog Flush" queue.
7. Since "Commit Order" queue is not empty (there is applier thread 2 in the
queue), it enters the conditional wait `m_stage_cond_leader` with an
intention to become the leader for both the "Binlog Flush" and
"Commit Order" queues.
8. Applier thread 1 releases the lock MYSQL_BIN_LOG::LOCK_log and proceeds to update
the GTID by calling gtid_state->update_commit_group() from
Commit_order_manager::flush_engine_and_signal_threads().
9. Applier thread 2 acquires the lock MYSQL_BIN_LOG::LOCK_log.
9.1. It checks if there is any thread waiting in the "Binlog Flush" queue
to become the leader. Here it finds the client thread waiting to be
the leader.
9.2. It releases the lock MYSQL_BIN_LOG::LOCK_log and signals on the
cond_var `m_stage_cond_leader` and enters a conditional wait until the
thread's `tx_commit_pending` is set to false by the client thread
(will be done in the
Commit_stage_manager::process_final_stage_for_ordered_commit_group()
called by client thread from fetch_and_process_flush_stage_queue()).
10. The client thread wakes up from the cond_var `m_stage_cond_leader`. The
thread has now become a leader and it is its responsibility to update GTID
of applier thread 2.
10.1. It acquires the lock MYSQL_BIN_LOG::LOCK_log.
10.2. Returns from `enroll_for()` and proceeds to process the
"Commit Order" and "Binlog Flush" queues.
10.3. Fetches the "Commit Order" and "Binlog Flush" queues.
10.4. Performs the storage engine flush by calling ha_flush_logs() from
fetch_and_process_flush_stage_queue().
10.5. Proceeds to update the GTID of threads in "Commit Order" queue by
calling gtid_state->update_commit_group() from
Commit_stage_manager::process_final_stage_for_ordered_commit_group().
11. At this point, we will have
- Client thread performing GTID update on behalf if applier thread 2 (from step 10.5), and
- Applier thread 1 performing GTID update for itself (from step 8).
Due to the lack of proper synchronization between the above two threads,
there exists a time window where both threads can call
gtid_state->update_commit_group() concurrently.
In subsequent steps, both threads simultaneously try to modify the contents
of the array `commit_group_sidnos` which is used to track the lock status of
sidnos. This concurrent access to `update_commit_group()` can cause a
lock-leak resulting in one thread acquiring the sidno lock and not
releasing at all.
-----------------------------------------------------------------------------------------------------------
Client thread Applier Thread 1
-----------------------------------------------------------------------------------------------------------
update_commit_group() => global_sid_lock->rdlock(); update_commit_group() => global_sid_lock->rdlock();
calls update_gtids_impl_lock_sidnos() calls update_gtids_impl_lock_sidnos()
set commit_group_sidno[2] = true set commit_group_sidno[2] = true
lock_sidno(2) -> successful
lock_sidno(2) -> waits
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) {
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Applier thread continues..
lock_sidno(2) -> successful
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) { <=== this check fails and lock is not released.
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Client thread continues without releasing the lock
-----------------------------------------------------------------------------------------------------------
12. As the above lock-leak can also happen the other way i.e, the applier
thread fails to unlock, there can be different consequences hereafter.
13. If the client thread continues without releasing the lock, then at a later
stage, it can enter into a deadlock with the applier thread performing a
GTID update with stack trace.
Client_thread
-------------
#1 __GI___lll_lock_wait
#2 ___pthread_mutex_lock
#3 native_mutex_lock <= waits for commit lock while holding sidno lock
#4 Commit_stage_manager::enroll_for
#5 MYSQL_BIN_LOG::change_stage
#6 MYSQL_BIN_LOG::ordered_commit
#7 MYSQL_BIN_LOG::commit
#8 ha_commit_trans
#9 trans_commit_implicit
#10 mysql_create_like_table
#11 Sql_cmd_create_table::execute
#12 mysql_execute_command
percona#13 dispatch_sql_command
Applier thread
--------------
#1 ___pthread_mutex_lock
#2 native_mutex_lock
#3 safe_mutex_lock
#4 Gtid_state::update_gtids_impl_lock_sidnos <= waits for sidno lock
#5 Gtid_state::update_commit_group
#6 Commit_order_manager::flush_engine_and_signal_threads <= acquires commit lock here
#7 Commit_order_manager::finish
#8 Commit_order_manager::wait_and_finish
#9 ha_commit_low
#10 trx_coordinator::commit_in_engines
#11 MYSQL_BIN_LOG::commit
#12 ha_commit_trans
percona#13 trans_commit
percona#14 Xid_log_event::do_commit
percona#15 Xid_apply_log_event::do_apply_event_worker
percona#16 Slave_worker::slave_worker_exec_event
percona#17 slave_worker_exec_job_group
percona#18 handle_slave_worker
14. If the applier thread continues without releasing the lock, then at a later
stage, it can perform recursive locking while setting the GTID for the next
transaction (in set_gtid_next()).
In debug builds the above case hits the assertion
`safe_mutex_assert_not_owner()` meaning the lock is already acquired by the
replica applier thread when it tries to re-acquire the lock.
Solution
--------
In the above problematic example, when seen from each thread
individually, we can conclude that there is no problem in the order of lock
acquisition, thus there is no need to change the lock order.
However, the root cause for this problem is that multiple threads can
concurrently access to the array `Gtid_state::commit_group_sidnos`.
In its initial implementation, it was expected that threads should
hold the `MYSQL_BIN_LOG::LOCK_commit` before modifying its contents. But it
was not considered when upstream implemented WL#7846 (MTS:
slave-preserve-commit-order when log-slave-updates/binlog is disabled).
With this patch, we now ensure that `MYSQL_BIN_LOG::LOCK_commit` is acquired
when the client thread (binlog flush leader) when it tries to perform GTID
update on behalf of threads waiting in "Commit Order" queue, thus providing a
guarantee that `Gtid_state::commit_group_sidnos` array is never accessed
without the protection of `MYSQL_BIN_LOG::LOCK_commit`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.