Add _login and _hostname as connection attribute. #10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This works on UNIX, didn't test on Windows |
|
Hi, thank you for your contribution. Please confirm this code is submitted under the terms of the OCA (Oracle's Contribution Agreement) you have previously signed by cutting and pasting the following text as a comment: |
|
I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it. |
|
Hi, thank you for your contribution. Your code has been assigned to an internal queue. Please follow |
bjornmu
pushed a commit
that referenced
this pull request
Feb 5, 2016
…QL_CMD)-> GET_XA_OPT()
Bug#22273964 INNODB: FAILING ASSERTION: TOTAL_TRX >= TRX_SYS->N_PREPARED_TRX
**Problem description**
An assertion of Bug#21942487
static_cast<Sql_cmd_xa_commit*>(thd->lex->m_sql_cmd)->
get_xa_opt() == XA_ONE_PHASE
#8 0x0000000000e3a5e1 in ha_commit_low
#9 0x00000000015158e6 in TC_LOG_DUMMY::commit
#10 0x0000000001529bc3 in Sql_cmd_xa_commit::trans_xa_commit
was caused by incorrect assumption by XA binlogging of wl6860 in
that @@session.pseudo_slave_mode can be only set 1 through binlog.
If fact the var can be set so manually.
Another assert in Bug#22273964 takes place for the very same reason.
**Solution**
The most reliable way to identify that the executing thread
is a binlog applier must include both the checking of rli_fake
and @@session.pseudo_slave_mode.
The former merely checking of the session variable as atttemp to
identify the binlog applier is replaced by checking the conjuction
of the two properties.
Notice that load containting SET @@session.pseudo_slave_mode=1 and
BINLOG '' pseudo-queries is not necessary authentic to mysqlbinlog output.
Nevertheless it must be processable when it's manually engineered such way.
A test is included to prove that as well.
**Note**
Beware of Bug #19502202 SERVER SHUTDOWN HANG SEEN IN SOME INNODB & RPL TESTS
at testing with binlog.binlog_xa_prepared_disconnect that still fails sporadically.
bjornmu
pushed a commit
that referenced
this pull request
Apr 10, 2017
Patch #10: Fix -Wunused-parameter warnings in sp*
bjornmu
pushed a commit
that referenced
this pull request
Apr 10, 2017
…FOR DATABASE Patch #10 To repeat: ./mtr --charset-for-testdb=utf8mb4 --defaults-file=include/utf8mb4_my.cnf ctype_collate explain_json_all explain_json_none func_bitwise_ops func_compress func_like innodb_pk_extension_off innodb_pk_extension_on insert_select select_found select_safe subquery_mat subquery_mat_all subquery_mat_none Change-Id: Ic4d6491daea4e87157ea9373394462174b70b566
pobrzut
pushed a commit
that referenced
this pull request
May 8, 2017
pobrzut
pushed a commit
that referenced
this pull request
May 8, 2017
bjornmu
pushed a commit
that referenced
this pull request
Sep 21, 2017
Patch #10: Split up seek_no_dup_elimination() in multiple helper functions (one per path leg type) to avoid having to go through the same switch statement over and over again when processing wildcards or ellipses. In json_binary::Value, make all the member variables non-const. This allows us to skip initialization of member variables that are not used. It also allows us to remove the custom assignment operator that works around the constness. Microbenchmarks (64-bit, Intel Core i7-4770 3.4 GHz, GCC 6.3): BM_JsonBinarySearchEllipsis 8902 ns/iter [+89.8%] BM_JsonBinarySearchEllipsis_OnlyOne 80 ns/iter [ -7.5%] BM_JsonBinarySearchKey 72 ns/iter [ -6.9%] Change-Id: I975da622899ff2eddc5505d9d9759996bf3d878e
surbhat1595
pushed a commit
that referenced
this pull request
Apr 25, 2019
…E TO A SERVER
Problem
========================================================================
Running the GCS tests with ASAN seldomly reports a user-after-free of
the server reference that the acceptor_learner_task uses.
Here is an excerpt of ASAN's output:
==43936==ERROR: AddressSanitizer: heap-use-after-free on address 0x63100021c840 at pc 0x000000530ff8 bp 0x7fc0427e8530 sp 0x7fc0427e8520
WRITE of size 8 at 0x63100021c840 thread T3
#0 0x530ff7 in server_detected /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:962
#1 0x533814 in buffered_read_bytes /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1249
#2 0x5481af in buffered_read_msg /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1399
#3 0x51e171 in acceptor_learner_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4690
#4 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140
#5 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324
#6 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164
#7 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107
#8 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4)
#9 0x7fc047ff2bfc in __clone (/lib64/libc.so.6+0xfebfc)
0x63100021c840 is located 64 bytes inside of 65688-byte region [0x63100021c800,0x63100022c898)
freed by thread T3 here:
#0 0x7fc04a5d7508 in __interceptor_free (/lib64/libasan.so.4+0xde508)
#1 0x52cf86 in freesrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:836
#2 0x52ea78 in srv_unref /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:868
#3 0x524c30 in reply_handler_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4914
#4 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140
#5 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324
#6 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164
#7 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107
#8 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4)
previously allocated by thread T3 here:
#0 0x7fc04a5d7a88 in __interceptor_calloc (/lib64/libasan.so.4+0xdea88)
#1 0x543604 in mksrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:721
#2 0x543b4c in addsrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:755
#3 0x54af61 in update_servers /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1747
#4 0x501082 in site_install_action /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1572
#5 0x55447c in import_config /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/site_def.c:486
#6 0x506dfc in handle_x_snapshot /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:5257
#7 0x50c444 in xcom_fsm /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:5325
#8 0x516c36 in dispatch_op /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4510
#9 0x521997 in acceptor_learner_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4772
#10 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140
#11 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324
#12 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164
#13 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107
#14 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4)
Analysis
========================================================================
The server structure is reference counted by the associated sender_task
and reply_handler_task.
When they finish, they unreference the server, which leads to its memory
being freed.
However, the acceptor_learner_task keeps a "naked" reference to the
server structure.
Under the right ordering of operations, i.e. the sender_task and
reply_handler_task terminating after the acceptor_learner_task acquires,
but before it uses, the reference to the server structure, leads to the
acceptor_learner_task accessing the server structure after it has been
freed.
Solution
========================================================================
Let the acceptor_learner_task also reference count the server structure
so it is not freed while still in use.
Reviewed-by: André Negrão <andre.negrao@oracle.com>
Reviewed-by: Venkatesh Venugopal <venkatesh.venugopal@oracle.com>
RB: 21209
bjornmu
pushed a commit
that referenced
this pull request
Apr 20, 2021
…oclose] src\ndbapi\ndb_cluster_connection.cpp(473,3): warning C4302: 'type cast': truncation from 'Ndb_cluster_connection_impl *' to 'long' src\ndbapi\ndb_cluster_connection.cpp(473,3): warning C4311: 'type cast': pointer truncation from 'Ndb_cluster_connection_impl *' to 'long' Change-Id: Iab757f4e868fee7fcf441f09b40d007bf7bbe05c
piki
added a commit
to planetscale/mysql-server
that referenced
this pull request
Apr 27, 2022
Fix the unit tests and generate a working container
dveeden
pushed a commit
to dveeden/mysql-server
that referenced
this pull request
May 5, 2023
Change-Id: I69245d3ffb847bfedf0c1abe478e94d069d015f8
bjornmu
pushed a commit
that referenced
this pull request
Oct 25, 2023
When a node starts up, it might connect to a management server to obtain the cluster configuration. The component that does this is known as ConfigRetriever. This patch adds an instance of TlsKeyManager to ConfigRetriever for use in MGM TLS by both sorts of ndbd processes: the data node and the angel process. The data node (but not the angel) also has a TlsKeyManager in the global TransporterRegistry. In the data node, these two TlsKeyManagers are initialized with the same node type and TLS search path, so they will both use the same certificate. In a data node, ConfigRetriever's MGM connection is the one that will get turned in to a transporter. In the existing MTR test tls_off_certs, the behavior changes so that the data nodes now have TLS connections to the MGM node. An NDB management server might also be a client of some other NDB management server. This patch adds support for the --ndb-mgm-tls option to ndb_mgmd, for setting its MGM TLS requirement level when it acts as a client. This patch extends the test testMgmd -n NdbdWithCertificate to run with [MGM]RequireTls set to true. Change-Id: I9ab68bef61b80a18edbf1375366c81ed874d7026
venkatesh-prasad-v
added a commit
to venkatesh-prasad-v/mysql-server
that referenced
this pull request
Jan 26, 2024
…and a local
DDL executed
https://bugs.mysql.com/bug.php?id=113727
Problem
-------
In high concurrency scenarios, MySQL replica can enter into a deadlock due to a
race condition between the replica applier thread and the client thread
performing a binlog group commit.
Analysis
--------
It needs at least 3 threads for this deadlock to happen
1. One client thread
2. Two replica applier threads
How this deadlock happens?
--------------------------
0. Binlog is enabled on replica, but log_replica_updates is disabled.
1. Initially, both "Commit Order" and "Binlog Flush" queues are empty.
2. Replica applier thread 1 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
3. Since both "Commit Order" and "Binlog Flush" queues are empty, the applier
thread 1
3.1. Becomes leader (In Commit_stage_manager::enroll_for()).
3.2. Registers in the commit order queue.
3.3. Acquires the lock MYSQL_BIN_LOG::LOCK_log.
3.4. Commit Order queue is emptied, but the lock MYSQL_BIN_LOG::LOCK_log is
not yet released.
NOTE: SE commit for applier thread is already done by the time it reaches
here.
4. Replica applier thread 2 enters the group commit pipeline to register in the
"Commit Order" queue since `log-replica-updates` is disabled on the replica
node.
5. Since the "Commit Order" queue is empty (emptied by applier thread 1 in 3.4), the
applier thread 2
5.1. Becomes leader (In Commit_stage_manager::enroll_for())
5.2. Registers in the commit order queue.
5.3. Tries to acquire the lock MYSQL_BIN_LOG::LOCK_log. Since it is held by applier
thread 1 it will wait until the lock is released.
6. Client thread enters the group commit pipeline to register in the
"Binlog Flush" queue.
7. Since "Commit Order" queue is not empty (there is applier thread 2 in the
queue), it enters the conditional wait `m_stage_cond_leader` with an
intention to become the leader for both the "Binlog Flush" and
"Commit Order" queues.
8. Applier thread 1 releases the lock MYSQL_BIN_LOG::LOCK_log and proceeds to update
the GTID by calling gtid_state->update_commit_group() from
Commit_order_manager::flush_engine_and_signal_threads().
9. Applier thread 2 acquires the lock MYSQL_BIN_LOG::LOCK_log.
9.1. It checks if there is any thread waiting in the "Binlog Flush" queue
to become the leader. Here it finds the client thread waiting to be
the leader.
9.2. It releases the lock MYSQL_BIN_LOG::LOCK_log and signals on the
cond_var `m_stage_cond_leader` and enters a conditional wait until the
thread's `tx_commit_pending` is set to false by the client thread
(will be done in the
Commit_stage_manager::process_final_stage_for_ordered_commit_group()
called by client thread from fetch_and_process_flush_stage_queue()).
10. The client thread wakes up from the cond_var `m_stage_cond_leader`. The
thread has now become a leader and it is its responsibility to update GTID
of applier thread 2.
10.1. It acquires the lock MYSQL_BIN_LOG::LOCK_log.
10.2. Returns from `enroll_for()` and proceeds to process the
"Commit Order" and "Binlog Flush" queues.
10.3. Fetches the "Commit Order" and "Binlog Flush" queues.
10.4. Performs the storage engine flush by calling ha_flush_logs() from
fetch_and_process_flush_stage_queue().
10.5. Proceeds to update the GTID of threads in "Commit Order" queue by
calling gtid_state->update_commit_group() from
Commit_stage_manager::process_final_stage_for_ordered_commit_group().
11. At this point, we will have
- Client thread performing GTID update on behalf if applier thread 2 (from step 10.5), and
- Applier thread 1 performing GTID update for itself (from step 8).
Due to the lack of proper synchronization between the above two threads,
there exists a time window where both threads can call
gtid_state->update_commit_group() concurrently.
In subsequent steps, both threads simultaneously try to modify the contents
of the array `commit_group_sidnos` which is used to track the lock status of
sidnos. This concurrent access to `update_commit_group()` can cause a
lock-leak resulting in one thread acquiring the sidno lock and not
releasing at all.
-----------------------------------------------------------------------------------------------------------
Client thread Applier Thread 1
-----------------------------------------------------------------------------------------------------------
update_commit_group() => global_sid_lock->rdlock(); update_commit_group() => global_sid_lock->rdlock();
calls update_gtids_impl_lock_sidnos() calls update_gtids_impl_lock_sidnos()
set commit_group_sidno[2] = true set commit_group_sidno[2] = true
lock_sidno(2) -> successful
lock_sidno(2) -> waits
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) {
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Applier thread continues..
lock_sidno(2) -> successful
update_gtids_impl_own_gtid() -> Add the thd->owned_gtid in `executed_gtids()`
if (commit_group_sidnos[2]) { <=== this check fails and lock is not released.
unlock_sidno(2);
commit_group_sidnos[2] = false;
}
Client thread continues without releasing the lock
-----------------------------------------------------------------------------------------------------------
12. As the above lock-leak can also happen the other way i.e, the applier
thread fails to unlock, there can be different consequences hereafter.
13. If the client thread continues without releasing the lock, then at a later
stage, it can enter into a deadlock with the applier thread performing a
GTID update with stack trace.
Client_thread
-------------
mysql#1 __GI___lll_lock_wait
mysql#2 ___pthread_mutex_lock
mysql#3 native_mutex_lock <= waits for commit lock while holding sidno lock
mysql#4 Commit_stage_manager::enroll_for
mysql#5 MYSQL_BIN_LOG::change_stage
mysql#6 MYSQL_BIN_LOG::ordered_commit
mysql#7 MYSQL_BIN_LOG::commit
mysql#8 ha_commit_trans
mysql#9 trans_commit_implicit
mysql#10 mysql_create_like_table
mysql#11 Sql_cmd_create_table::execute
mysql#12 mysql_execute_command
mysql#13 dispatch_sql_command
Applier thread
--------------
mysql#1 ___pthread_mutex_lock
mysql#2 native_mutex_lock
mysql#3 safe_mutex_lock
mysql#4 Gtid_state::update_gtids_impl_lock_sidnos <= waits for sidno lock
mysql#5 Gtid_state::update_commit_group
mysql#6 Commit_order_manager::flush_engine_and_signal_threads <= acquires commit lock here
mysql#7 Commit_order_manager::finish
mysql#8 Commit_order_manager::wait_and_finish
mysql#9 ha_commit_low
mysql#10 trx_coordinator::commit_in_engines
mysql#11 MYSQL_BIN_LOG::commit
mysql#12 ha_commit_trans
mysql#13 trans_commit
mysql#14 Xid_log_event::do_commit
mysql#15 Xid_apply_log_event::do_apply_event_worker
mysql#16 Slave_worker::slave_worker_exec_event
mysql#17 slave_worker_exec_job_group
mysql#18 handle_slave_worker
14. If the applier thread continues without releasing the lock, then at a later
stage, it can perform recursive locking while setting the GTID for the next
transaction (in set_gtid_next()).
In debug builds the above case hits the assertion
`safe_mutex_assert_not_owner()` meaning the lock is already acquired by the
replica applier thread when it tries to re-acquire the lock.
Solution
--------
In the above problematic example, when seen from each thread
individually, we can conclude that there is no problem in the order of lock
acquisition, thus there is no need to change the lock order.
However, the root cause for this problem is that multiple threads can
concurrently access to the array `Gtid_state::commit_group_sidnos`.
In its initial implementation, it was expected that threads should
hold the `MYSQL_BIN_LOG::LOCK_commit` before modifying its contents. But it
was not considered when upstream implemented WL#7846 (MTS:
slave-preserve-commit-order when log-slave-updates/binlog is disabled).
With this patch, we now ensure that `MYSQL_BIN_LOG::LOCK_commit` is acquired
when the client thread (binlog flush leader) when it tries to perform GTID
update on behalf of threads waiting in "Commit Order" queue, thus providing a
guarantee that `Gtid_state::commit_group_sidnos` array is never accessed
without the protection of `MYSQL_BIN_LOG::LOCK_commit`.
bjornmu
pushed a commit
that referenced
this pull request
Jul 1, 2024
… for connection xxx'.
The new iterator based explains are not impacted.
The issue here is a race condition. More than one thread is using the
query term iterator at the same time (whoch is neithe threas safe nor
reantrant), and part of its state is in the query terms being visited
which leads to interference/race conditions.
a) the explain thread
uses an iterator here:
Sql_cmd_explain_other_thread::execute
is inspecting the Query_expression of the running query
calling master_query_expression()->find_blocks_query_term which uses
an iterator over the query terms in the query expression:
for (auto qt : query_terms<>()) {
if (qt->query_block() == qb) {
return qt;
}
}
the above search fails to find qb due to the interference of the
thread b), see below, and then tries to access a nullpointer:
* thread #36, name = ‘connection’, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x000000010bb3cf0d mysqld`Query_block::type(this=0x00007f8f82719088) const at sql_lex.cc:4441:11
frame #1: 0x000000010b83763e mysqld`(anonymous namespace)::Explain::explain_select_type(this=0x00007000020611b8) at opt_explain.cc:792:50
frame #2: 0x000000010b83cc4d mysqld`(anonymous namespace)::Explain_join::explain_select_type(this=0x00007000020611b8) at opt_explain.cc:1487:21
frame #3: 0x000000010b837c34 mysqld`(anonymous namespace)::Explain::prepare_columns(this=0x00007000020611b8) at opt_explain.cc:744:26
frame #4: 0x000000010b83ea0e mysqld`(anonymous namespace)::Explain_join::explain_qep_tab(this=0x00007000020611b8, tabnum=0) at opt_explain.cc:1415:32
frame #5: 0x000000010b83ca0a mysqld`(anonymous namespace)::Explain_join::shallow_explain(this=0x00007000020611b8) at opt_explain.cc:1364:9
frame #6: 0x000000010b83379b mysqld`(anonymous namespace)::Explain::send(this=0x00007000020611b8) at opt_explain.cc:770:14
frame #7: 0x000000010b834147 mysqld`explain_query_specification(explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00, query_term=0x00007f8f82719088, ctx=CTX_JOIN) at opt_explain.cc:2088:20
frame #8: 0x000000010bd36b91 mysqld`Query_expression::explain_query_term(this=0x00007f8f7a090360, explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00, qt=0x00007f8f82719088) at sql_union.cc:1519:11
frame #9: 0x000000010bd36c68 mysqld`Query_expression::explain_query_term(this=0x00007f8f7a090360, explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00, qt=0x00007f8f8271d748) at sql_union.cc:1526:13
frame #10: 0x000000010bd373f7 mysqld`Query_expression::explain(this=0x00007f8f7a090360, explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00) at sql_union.cc:1591:7
frame #11: 0x000000010b835820 mysqld`mysql_explain_query_expression(explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00, unit=0x00007f8f7a090360) at opt_explain.cc:2392:17
frame #12: 0x000000010b835400 mysqld`explain_query(explain_thd=0x00007f8fbb111e00, query_thd=0x00007f8fbb919c00, unit=0x00007f8f7a090360) at opt_explain.cc:2353:13
* frame #13: 0x000000010b8363e4 mysqld`Sql_cmd_explain_other_thread::execute(this=0x00007f8fba585b68, thd=0x00007f8fbb111e00) at opt_explain.cc:2531:11
frame #14: 0x000000010bba7d8b mysqld`mysql_execute_command(thd=0x00007f8fbb111e00, first_level=true) at sql_parse.cc:4648:29
frame #15: 0x000000010bb9e230 mysqld`dispatch_sql_command(thd=0x00007f8fbb111e00, parser_state=0x0000700002065de8) at sql_parse.cc:5303:19
frame #16: 0x000000010bb9a4cb mysqld`dispatch_command(thd=0x00007f8fbb111e00, com_data=0x0000700002066e38, command=COM_QUERY) at sql_parse.cc:2135:7
frame #17: 0x000000010bb9c846 mysqld`do_command(thd=0x00007f8fbb111e00) at sql_parse.cc:1464:18
frame #18: 0x000000010b2f2574 mysqld`handle_connection(arg=0x0000600000e34200) at connection_handler_per_thread.cc:304:13
frame #19: 0x000000010e072fc4 mysqld`pfs_spawn_thread(arg=0x00007f8fba8160b0) at pfs.cc:3051:3
frame #20: 0x00007ff806c2b202 libsystem_pthread.dylib`_pthread_start + 99
frame #21: 0x00007ff806c26bab libsystem_pthread.dylib`thread_start + 15
b) the query thread being explained is itself performing LEX::cleanup
and as part of the iterates over the query terms, but still allows
EXPLAIN of the query plan since
thd->query_plan.set_query_plan(SQLCOM_END, ...)
hasn't been called yet.
20:frame: Query_terms<(Visit_order)1, (Visit_leaves)0>::Query_term_iterator::operator++() (in mysqld) (query_term.h:613)
21:frame: Query_expression::cleanup(bool) (in mysqld) (sql_union.cc:1861)
22:frame: LEX::cleanup(bool) (in mysqld) (sql_lex.h:4286)
30:frame: Sql_cmd_dml::execute(THD*) (in mysqld) (sql_select.cc:799)
31:frame: mysql_execute_command(THD*, bool) (in mysqld) (sql_parse.cc:4648)
32:frame: dispatch_sql_command(THD*, Parser_state*) (in mysqld) (sql_parse.cc:5303)
33:frame: dispatch_command(THD*, COM_DATA const*, enum_server_command) (in mysqld) (sql_parse.cc:2135)
34:frame: do_command(THD*) (in mysqld) (sql_parse.cc:1464)
57:frame: handle_connection(void*) (in mysqld) (connection_handler_per_thread.cc:304)
58:frame: pfs_spawn_thread(void*) (in mysqld) (pfs.cc:3053)
65:frame: _pthread_start (in libsystem_pthread.dylib) + 99
66:frame: thread_start (in libsystem_pthread.dylib) + 15
Solution:
This patch solves the issue by removing iterator state from
Query_term, making the query_term iterators thread safe. This solution
labels every child query_term with its index in its parent's
m_children vector. The iterator can therefore easily compute the next
child to visit based on Query_term::m_sibling_idx.
A unit test case is added to check reentrancy.
One can also manually verify that we have no remaining race condition
by running two client connections files (with \. <file>) with a big
number of copies of the repro query in one connection and a big number
of EXPLAIN format=json FOR <connection>, e.g.
EXPLAIN FORMAT=json FOR CONNECTION 8\G
in the other. The actual connection number would need to verified
in connection one, of course.
Change-Id: Ie7d56610914738ccbbecf399ccc4f465f7d26ea7
bjornmu
pushed a commit
that referenced
this pull request
Jul 1, 2024
…_into_hash_map() [1/2 noclose] To see this error, we needed to run with ASAN compiled in and --sanitize on the repro test case. It didn't trigger an ASAN error as such, but hit an assert. Possibly this happens only with ASAN because space usage increases on the heap due to ASAN padding before and after allocated objects. The error is another instance of the third issue solved in Bug#35686098 Assertion `n < size()' failed in Element_type& Mem_root_array_YY: (quote): "we ran out of space in the dedicated hash table upon re-reading one of the on-disk chunks into the hash table." The fix there was to use the general mem_root instead of the dedicated one for the hash set operation for that particular corner case. Specifically, we used the general mem_root when allocating the hash map *payload*. In this bug, we see the same issue: we can't fit the last row (#10 out of 10 long varchar strings, i.e. blobs in the repro) into the hash table mem_root: this time we run out when we allocate space for the hash map *key*, in check_unique_fields_hash_map prior to calling ImmutableStringWithLength::Encode(<hash key>). The solution is the same, use the general mem_root for this case: i.e. we generalize the solution from Bug#35686098 to cover running out of space for both the key and payload. Change-Id: Id869e6afad7c996950779c008dcf650e21b17027
gipulla
pushed a commit
that referenced
this pull request
Oct 15, 2024
…_into_hash_map() [1/2 noclose] To see this error, we needed to run with ASAN compiled in and --sanitize on the repro test case. It didn't trigger an ASAN error as such, but hit an assert. Possibly this happens only with ASAN because space usage increases on the heap due to ASAN padding before and after allocated objects. The error is another instance of the third issue solved in Bug#35686098 Assertion `n < size()' failed in Element_type& Mem_root_array_YY: (quote): "we ran out of space in the dedicated hash table upon re-reading one of the on-disk chunks into the hash table." The fix there was to use the general mem_root instead of the dedicated one for the hash set operation for that particular corner case. Specifically, we used the general mem_root when allocating the hash map *payload*. In this bug, we see the same issue: we can't fit the last row (#10 out of 10 long varchar strings, i.e. blobs in the repro) into the hash table mem_root: this time we run out when we allocate space for the hash map *key*, in check_unique_fields_hash_map prior to calling ImmutableStringWithLength::Encode(<hash key>). The solution is the same, use the general mem_root for this case: i.e. we generalize the solution from Bug#35686098 to cover running out of space for both the key and payload. Change-Id: Id869e6afad7c996950779c008dcf650e21b17027
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://bugs.mysql.com/bug.php?id=77190