Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

InnoDB assertion around sync_thread_levels with --wsrep_log_conflicts=ON #28

Closed
philip-galera opened this issue Dec 15, 2014 · 4 comments
Closed

InnoDB assertion around sync_thread_levels with --wsrep_log_conflicts=ON #28

philip-galera opened this issue Dec 15, 2014 · 4 comments
Labels
bug

Comments

@philip-galera
Copy link
Contributor

When attempting to run a simple test with --wsrep_log_conflicts=ON, the slaveasserted as follows:

2014-12-15 11:41:28 6517 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
InnoDB: sync levels should be > 298 but a level is 297
Mutex created at trx0trx.cc 108
InnoDB: Locked mutex: addr 0x7fd4580117f0 thread 140550279898880 file /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc line 2500
InnoDB: sync_thread_levels_g(array, 298) does not hold!
2014-12-15 11:41:29 7fd46981a700  InnoDB: Assertion failure in thread 140550279898880 in file sync0sync.cc line 1172

#6  0x0000000000cee91c in sync_thread_add_level (latch=0x29c0c48, level=298, relock=0) at /home/philips/git/mysql-wsrep/storage/innobase/sync/sync0sync.cc:1172
#7  0x0000000000cedb83 in mutex_set_debug_info (mutex=0x29c0c48, file_name=0x10429c0 "/home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc", line=1650) at /home/philips/git/mysql-wsrep/storage/innobase/sync/sync0sync.cc:615
#8  0x0000000000be395a in mutex_enter_func (mutex=0x29c0c48, file_name=0x10429c0 "/home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc", line=1650) at /home/philips/git/mysql-wsrep/storage/innobase/include/sync0sync.ic:226
#9  0x0000000000be39e5 in pfs_mutex_enter_func (mutex=0x29c0c48, file_name=0x10429c0 "/home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc", line=1650) at /home/philips/git/mysql-wsrep/storage/innobase/include/sync0sync.ic:257
#10 0x0000000000beb1c3 in wsrep_kill_victim (trx=0x7fd4580117e8, lock=0x7fd438079780) at /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc:1650
#11 0x0000000000beb41f in lock_rec_other_has_conflicting (mode=1027, block=0x7fd46859b040, heap_no=2, trx=0x7fd4580117e8) at /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc:1714
#12 0x0000000000bed243 in lock_rec_lock_slow (impl=0, mode=1027, block=0x7fd46859b040, heap_no=2, index=0x7fd45803d258, thr=0x7fd4580104b0) at /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc:2508
#13 0x0000000000bed574 in lock_rec_lock (impl=0, mode=1027, block=0x7fd46859b040, heap_no=2, index=0x7fd45803d258, thr=0x7fd4580104b0) at /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc:2592
#14 0x0000000000bf7656 in lock_clust_rec_read_check_and_lock (flags=0, block=0x7fd46859b040, rec=0x7fd46886407e "текÑ\302\201Ñ‚", index=0x7fd45803d258, offsets=0x7fd4698167a0, mode=LOCK_X, gap_mode=1024, thr=0x7fd4580104b0) at /home/philips/git/mysql-wsrep/storage/innobase/lock/lock0lock.cc:6774
#15 0x0000000000cba9a5 in sel_set_rec_lock (block=0x7fd46859b040, rec=0x7fd46886407e "текÑ\302\201Ñ‚", index=0x7fd45803d258, offsets=0x7fd4698167a0, mode=3, type=1024, thr=0x7fd4580104b0) at /home/philips/git/mysql-wsrep/storage/innobase/row/row0sel.cc:1011
#16 0x0000000000cc10df in row_search_for_mysql (buf=0x7fd458042070 "\026", mode=2, prebuilt=0x7fd45800f878, match_mode=1, direction=0) at /home/philips/git/mysql-wsrep/storage/innobase/row/row0sel.cc:4499
#17 0x0000000000b9bf75 in ha_innobase::index_read (this=0x7fd45803a0f0, buf=0x7fd458042070 "\026", key_ptr=0x7fd458043210 "\026", key_len=767, find_flag=HA_READ_KEY_EXACT) at /home/philips/git/mysql-wsrep/storage/innobase/handler/ha_innodb.cc:8218
#18 0x0000000000b9cf6c in ha_innobase::rnd_pos (this=0x7fd45803a0f0, buf=0x7fd458042070 "\026", pos=0x7fd458043210 "\026") at /home/philips/git/mysql-wsrep/storage/innobase/handler/ha_innodb.cc:8720
#19 0x0000000000658c04 in handler::ha_rnd_pos (this=0x7fd45803a0f0, buf=0x7fd458042070 "\026", pos=0x7fd458043210 "\026") at /home/philips/git/mysql-wsrep/sql/handler.cc:2786
#20 0x0000000000663f5b in handler::rnd_pos_by_record (this=0x7fd45803a0f0, record=0x7fd458042070 "\026") at /home/philips/git/mysql-wsrep/sql/handler.h:2325
#21 0x0000000000a2423b in Rows_log_event::do_index_scan_and_update (this=0x7fd458045380, rli=0x7fd45800bc20) at /home/philips/git/mysql-wsrep/sql/log_event.cc:10540
#22 0x0000000000a2640d in Rows_log_event::do_apply_event (this=0x7fd458045380, rli=0x7fd45800bc20) at /home/philips/git/mysql-wsrep/sql/log_event.cc:11362
#23 0x0000000000a11ad5 in Log_event::apply_event (this=0x7fd458045380, rli=0x7fd45800bc20) at /home/philips/git/mysql-wsrep/sql/log_event.cc:3112
#24 0x0000000000649d45 in wsrep_apply_events (thd=0x7fd4580009a0, events_buf=0x7fd46c9848bb, buf_len=0) at /home/philips/git/mysql-wsrep/sql/wsrep_applier.cc:151
#25 0x000000000064a32d in wsrep_apply_cb (ctx=0x7fd4580009a0, buf=0x7fd46c9848bb, buf_len=199, flags=1, meta=0x7fd469818b80) at /home/philips/git/mysql-wsrep/sql/wsrep_applier.cc:238
#26 0x00007fd474b5ecd9 in galera::TrxHandle::apply (this=this@entry=0x7fd458004680, recv_ctx=recv_ctx@entry=0x7fd4580009a0, apply_cb=apply_cb@entry=0x64a141 <wsrep_apply_cb(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*)>, meta=...) at galera/src/trx_handle.cpp:315
#27 0x00007fd474b9f43f in apply_trx_ws (recv_ctx=recv_ctx@entry=0x7fd4580009a0, apply_cb=0x64a141 <wsrep_apply_cb(void*, void const*, unsigned long, unsigned int, wsrep_trx_meta const*)>, commit_cb=0x64a6fe <wsrep_commit_cb(void*, unsigned int, wsrep_trx_meta const*, bool*, bool)>, trx=..., meta=...) at galera/src/replicator_smm.cpp:37
#28 0x00007fd474ba22a3 in galera::ReplicatorSMM::apply_trx (this=this@entry=0x281b9c0, recv_ctx=recv_ctx@entry=0x7fd4580009a0, trx=trx@entry=0x7fd458004680) at galera/src/replicator_smm.cpp:433
#29 0x00007fd474ba583e in galera::ReplicatorSMM::process_trx (this=0x281b9c0, recv_ctx=0x7fd4580009a0, trx=0x7fd458004680) at galera/src/replicator_smm.cpp:1223
#30 0x00007fd474b80b19 in galera::GcsActionSource::dispatch (this=this@entry=0x281c048, recv_ctx=recv_ctx@entry=0x7fd4580009a0, act=..., exit_loop=@0x7fd46981949c: false) at galera/src/gcs_action_source.cpp:116
#31 0x00007fd474b81b82 in galera::GcsActionSource::process (this=0x281c048, recv_ctx=0x7fd4580009a0, exit_loop=@0x7fd46981949c: false) at galera/src/gcs_action_source.cpp:181
#32 0x00007fd474ba5ed3 in galera::ReplicatorSMM::async_recv (this=0x281b9c0, recv_ctx=0x7fd4580009a0) at galera/src/replicator_smm.cpp:355
#33 0x000000000064bc24 in wsrep_replication_process (thd=0x7fd4580009a0) at /home/philips/git/mysql-wsrep/sql/wsrep_thd.cc:309
#34 0x000000000062a99a in start_wsrep_THD (arg=0x64bb6c <wsrep_replication_process(THD*)>) at /home/philips/git/mysql-wsrep/sql/mysqld.cc:5376
#35 0x0000003e90c0753a in start_thread () from /lib64/libpthread.so.0
#36 0x0000003e905045fd in clone () from /lib64/libc.so.6
@philip-galera
Copy link
Contributor Author

To reproduce:

 perl mysql-test-run.pl --suite galera galera_unicode_pk --mysqld=--wsrep_log_conflicts=ON

@ronin13
Copy link
Contributor

ronin13 commented Jan 5, 2015

This was fixed in PXC a while back in a wsrep-log-conflicts bug. Does the mtr test assert with PXC as well?

@philip-galera
Copy link
Contributor Author

@ronin13 the test should be available in your repository as well, you are welcome to try it.

@temeo temeo modified the milestone: 5.6-future Feb 3, 2015
@temeo temeo modified the milestone: 5.6.22-25.10 Feb 12, 2015
@philip-galera
Copy link
Contributor Author

This issue has now been fixed and the fix has been released.

SachinSetiya added a commit to MariaDB/server that referenced this issue Mar 14, 2017
@SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
0e0ae0b 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachinsetia1001@gmail.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Mar 23, 2017
@mariadb-SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
7a69fc6 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Mar 28, 2017
@mariadb-SachinSetiya @SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
2c22eab 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Apr 5, 2017
@mariadb-SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
a6b1cf1 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Apr 6, 2017
@mariadb-SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
5089fd0 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Apr 6, 2017
@mariadb-SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
cdd1dc8 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
SachinSetiya pushed a commit to MariaDB/server that referenced this issue Apr 7, 2017
@mariadb-SachinSetiya
MW-28, codership/mysql-wsrep#28 Fix sync_thread_levels debug assert
a992a51 
Introduced a new wsrep_trx_print_locking() which may be called
under lock_sys->mutex if the trx has locks.

Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ronin13 @temeo @philip-galera