Interesting label colors #1

midenok · 2016-10-05T01:36:59Z

No description provided.

This only merges MDEV-12253, adapting it to MDEV-12602 which is already present in 10.2 but not yet in the 10.1 revision that is being merged. TODO: Error handling in crash recovery needs to be improved. If a page cannot be decrypted (or read), we should cleanly abort the startup. If innodb_force_recovery is specified, we should ignore the problematic page and apply redo log to other pages. Currently, the test encryption.innodb-redo-badkey randomly fails like this (the last messages are from cmake -DWITH_ASAN): 2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption 2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown... i================================================================= ==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0 #0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750) #1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4 #2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17 #3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3 #4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2 #5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2 #6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3

…-12916 This is a joint patch fixing the following problems: MDEV-12875 Wrong VIEW column data type for COALESCE(int_column) MDEV-12886 Different default for INT and BIGINT column in a VIEW for a SELECT with ROLLUP MDEV-12916 Wrong column data type for an INT field of a cursor-anchored ROW variable All above problem happened because the global function ::create_tmp_field() called the top-level Item::create_tmp_field(), which made some tranformation for INT-result data types. For example, INT(11) became BIGINT(11), because 11 is a corner case and it's not known if it fits or does not fit into INT range, so Item::create_tmp_field() converted it to BIGINT(11) for safety. The main idea of this patch is to avoid such tranformations. 1. Fixing Item::create_tmp_field() not to have a special case for INT_RESULT. Item::create_tmp_field() is changed not to have a special case for INT_RESULT (which earlier made a decision based on Item's max_length). It now calls tmp_table_field_from_field_type() for INT_RESULT, therefore preserves the original data type (e.g. INT, YEAR) without conversion to BIGINT. This change is valid, because a number of recent fixes (e.g. in Item_func_int, Item_hybrid_func, Item_int, Item_splocal) guarantee that item->type_handler() now properly returns type_handler_long vs type_handler_longlong. So no adjustment by length is needed any more for Items returning INT_RESULT. After this change, Item::create_tmp_field() calls tmp_table_field_from_field_type() for all XXX_RESULT, except REAL_RESULT. 2. Fixing Item::create_tmp_field() not to have a special case for REAL_RESULT. Note, the reason for a special case for REAL_RESULT is to have a special constructor for Field_double(), forcing Field_real::not_fixed to be set to true. Taking into account that only Item_sum descendants actually need a special constructor call Field_double(not_fixed=true), not too loose precision when mixing individual rows to the aggregate result: - renaming Item::create_tmp_field() to Item_sum::create_tmp_field(). - changing Item::create_tmp_field() just to call tmp_table_field_from_field_type() for all XXX_RESULT types. A special case for REAL_RESULT in Item::create_tmp_field() is now gone. Item::create_tmp_field() is now symmetric for all XXX_RESULT types, and now just calls tmp_table_field_from_field_type(). 3. Fixing Item_func::create_field_for_create_select() not to have a special case for STRING_RESULT. After changes #1 and #2, the code in Item_func::create_field_for_create_select(), testing result_type(), becomes useless, because: now Item::create_tmp_field() and tmp_table_field_from_field_type() do exactly the same thing for all XXX_RESULT types for Item_func descendants: a. It calls tmp_table_field_from_field_type for STRING_RESULT directly. b. For other XXX_RESULT, it goes through Item::create_tmp_field(), which calls the global function ::create_tmp_field(), which calls item->create_tmp_field() for FUNC_ITEM, which calls tmp_table_field_from_field_type() again. So removing the virtual implementation of Item_func::create_field_for_create_select(). The inherited Item::create_field_for_create_select() now perfectly does the job, as it also calls tmp_table_field_from_field_type() for FUNC_ITEM, independently from XXX_RESULT type. 4. Taking into account #1 and #2, as well as some recent changes, removing virtual implementations: - Item_hybrid_func::create_tmp_field() - Item_hybrid_func::create_field_for_create_select() - Item_int_func::create_tmp_field() - Item_int_func::create_field_for_create_select() - Item_temporal_func::create_field_for_create_select() The derived versions from Item now perfectly work. 5. Moving a piece of code from create_tmp_field_from_item() to a new function create_tmp_field_from_item_finalize(), to reuse it in two places (see #6). 6. Changing the code responsible for BIT->INT/BIGIN tranformation (which is called for the cases when the created table, e.g. HEAP, does not fully support BIT) not to call create_tmp_field_from_item(), because the latter now calls tmp_table_field_from_field_type() instead of create_tmp_field() and thefore cannot do BIT transformation. So rewriting this code using a sequence of these calls: - item->type_handler_long_or_longlong() - handler->make_and_init_table_field() - create_tmp_field_from_item_finalize() 7. Miscelaneous changes: - Moving type_handler_long_or_longlong() from "protected" to "public", as it's now needed in the global function create_tmp_field(). 8. The above changes fixed MDEV-12875, MDEV-12886, MDEV-12916. So adding tests for these bugs.

…RT in subquery The bug happens because of a combination of unfortunate circumstances: 1. Arguments args[0] and args[2] of Item_func_concat point recursively (through Item_direct_view_ref's) to the same Item_func_conv_charset. Both args[0]->args[0]->ref[0] and args[2]->args[0]->ref[0] refer to this Item_func_conv_charset. 2. When Item_func_concat::args[0]->val_str() is called, Item_func_conv_charset::val_str() writes its result to Item_func_conc_charset::tmp_value. 3. Then, for optimization purposes (to avoid copying), Item_func_substr::val_str() initializes Item_func_substr::tmp_value to point to the buffer fragment owned by Item_func_conv_charset::tmp_value Item_func_substr::tmp_value is returned as a result of Item_func_concat::args[0]->val_str(). 4. Due to optimization to avoid memory reallocs, Item_func_concat::val_str() remembers the result of args[0]->val_str() in "res" and further uses "res" to collect the return value. 5. When Item_func_concat::args[2]->val_str() is called, Item_func_conv_charset::tmp_value gets overwritten (see #1), which effectively overwrites args[0]'s Item_func_substr::tmp_value (see #3), which effectively overwrites "res" (see #4). This patch does the following: a. Changes Item_func_conv_charset::val_str(String *str) to use tmp_value and str the other way around. After this change tmp_value is used to store a temporary result, while str is used to return the value. The fixes the second problem (without SUBSTR): SELECT CONCAT(t2,'-',t2) c2 FROM (SELECT CONVERT(t USING latin1) t2 FROM t1) sub; As Item_func_concat::val_str() supplies two different buffers when calling args[0]->val_str() and args[2]->val_str(), in the new reduction the result created during args[0]->val_str() does not get overwritten by args[2]->val_str(). b. Fixing the same problem in val_str() for similar classes Item_func_to_base64 Item_func_from_base64 Item_func_weight_string Item_func_hex Item_func_unhex Item_func_quote Item_func_compress Item_func_uncompress Item_func_des_encrypt Item_func_des_decrypt Item_func_conv_charset Item_func_reverse Item_func_soundex Item_func_aes_encrypt Item_func_aes_decrypt Item_func_buffer c. Fixing Item_func::val_str_from_val_str_ascii() the same way. Now Item_str_ascii_func::ascii_buff is used for temporary value, while the parameter passed to val_str() is used to return the result. This fixes the same problem when conversion (from ASCII to e.g. UCS2) takes place. See the ctype_ucs.test for example queries that returned wrong results before the fix. d. Some Item_func descendand classes had temporary String buffers (tmp_value and tmp_str), but did not really use them. Removing these temporary buffers from: Item_func_decode_histogram Item_func_format Item_func_binlog_gtid_pos Item_func_spatial_collection: e. Removing Item_func_buffer::tmp_value, because it's not used any more. f. Renaming Item_func_[un]compress::buffer to "tmp_value", for consistency with other classes. Note, this patch does not fix the following classes (although they have a similar problem): Item_str_conv Item_func_make_set Item_char_typecast They have a complex implementations and simple swapping between "tmp_value" and "str" won't work. These classes will be fixed separately.

CID 971836 (#1 of 1): Same on both sides (CONSTANT_EXPRESSION_RESULT) pointless_expression: The expression val != end && val != end does not accomplish anything because it evaluates to either of its identical operands, val != end.

Coverity report this as: CID 971840 (#1 of 1): Operands don't affect result (CONSTANT_EXPRESSION_RESULT) result_independent_of_operands: 4 | (flags & 1) is always true regardless of the values of its operands. This occurs as the logical first operand of "?:". The C order of precidence has | of higher precidence than ?:. The intenting implies an | of the 3 terms. Adjust to intented meaning.

Cherry-pick this fix from the upstream: commit 6ddedd8f1e0ddcbc24e8f9a005636c5463799ab7 Author: Sergei Petrunia <psergey@askmonty.org> Date: Tue Apr 10 11:43:01 2018 -0700 [mysql-5.6][PR] Issue MariaDB#802: MyRocks: Statement rollback doesnt work correctly for nesâ�¦ Summary: â�¦ted statements Variant #1: When the statement fails, we should roll back to the latest savepoint taken at the top level. Closes facebook/mysql-5.6#804 Differential Revision: D7509380 Pulled By: hermanlee fbshipit-source-id: 9a6f414

…ARY INCONSISTENCY The server crashes on a SELECT because of space id mismatch. The mismatch happens if the server crashes during an ALTER TABLE. There are actually two cases of inconsistency, and three fixes needed for the InnoDB problems. We have dictionary data (tablespace or table name) in 3 places: (a) The *.frm file is for the old table definition. (b) The InnoDB data dictionary is for the new table definition. (c) The file system did not rename the tablespace files yet. In this fix, we will not care if the *.frm file is in sync with the InnoDB data dictionary and file system. We will concentrate on the mismatch between (b) and (c). Two scenarios have been mentioned in this bug report. The simpler one first: 1. The changes to SYS_TABLES were committed, and MLOG_FILE_RENAME2 records were written in a single mini-transaction commit. The files were not yet renamed in the file system. 2a. The server is killed, without making a log checkpoint. 3a. The server refuses to start up, because replaying MLOG_FILE_RENAME2 fails. I failed to repeat this myself. I repeated step 3a with a saved dataset. The problem seems to be that MLOG_FILE_RENAME2 replay is incorrectly being skipped when there is no page-redo log or MLOG_FILE_NAME record for the old name of the tablespace. FIX#1: Recover the id-to-name mapping also from MLOG_FILE_RENAME2 records when scanning the redo log. It is not necessary to write MLOG_FILE_NAME records in addition to MLOG_FILE_RENAME2 records for renaming tablespace files. The scenario in the original Description involves a log checkpoint: 1. The changes to SYS_TABLES were committed, and MLOG_FILE_RENAME2 records were written in a single mini-transaction commit. 2. A log checkpoint and a server kill was injected. 3. Crash recovery will see no records (other than the MLOG_CHECKPOINT). 4. dict_check_tablespaces_and_store_max_id() will emit a message about a non-found table #sql-ib22*. 5. A mismatch is triggering the assertion failure. In my test, at step 4 the SYS_TABLES root page (0:8) contains these 3 records right before the page supremum: * delete-marked (committed) name=#sql-ib21* record, with space=10. * name=#sql-ib22*, space=9. * name=t1, space=10. space=10 is the rebuilt table (#sql-ib21*.ibd in the file system). space=9 is the old table (t1.ibd in the file system). The function dict_check_tablespaces_and_store_max_id() will enter t1.ibd with space_id=10 into the fil_system cache without noticing that t1.ibd contains space_id=9, because it invokes fil_open_single_table_tablespace() with validate=false. In MySQL 5.6, the space_id from all *.ibd files are being read when the redo log checkpoint LSN disagrees with the FIL_PAGE_FILE_FLUSH_LSN in the system tablespace. This field is only updated during a clean shutdown, after performing the final log checkpoint. FIX#2: dict_check_tablespaces_and_store_max_id() should pass validate=true to fil_open_single_table_tablespace() when a non-clean shutdown is detected, forcing the first page of each *.ibd file to be read. (We do not want to slow down startup after a normal shutdown.) With FIX#2, the SELECT would fail to find the table. This would introduce a regression, because before WL#7142, a copy of the table was accessible after recovery. FIX#3: Maintain a list of MLOG_FILE_RENAME2 records that have been written to the redo log, but not performed yet in the file system. When performing a checkpoint, re-emit these records to the redo log. In this way, a mismatch between (b) and (c) should be impossible. fil_name_process(): Refactored from fil_name_parse(). Adds an item to the id-to-filename mapping. fil_name_parse(): Parses and applies a MLOG_FILE_NAME, MLOG_FILE_DELETE or MLOG_FILE_RENAME2 record. This implements FIX#1. fil_name_write_rename(): A wrapper function for writing MLOG_FILE_RENAME2 records. fil_op_replay_rename(): Apply MLOG_FILE_RENAME2 records. Replaces fil_op_log_parse_or_replay(), whose logic was moved to fil_name_parse(). fil_tablespace_exists_in_mem(): Return fil_space_t* instead of bool. dict_check_tablespaces_and_store_max_id(): Add the parameter "validate" to implement FIX#2. log_sys->append_on_checkpoint: Extra log records to append in case of a checkpoint. Needed for FIX#3. log_append_on_checkpoint(): New function, to update log_sys->append_on_checkpoint. mtr_write_log(): New function, to append mtr_buf_t to the redo log. fil_names_clear(): Append the data from log_sys->append_on_checkpoint if needed. ha_innobase::commit_inplace_alter_table(): Add any MLOG_FILE_RENAME2 records to log_sys->append_on_checkpoint(), and remove them once the files have been renamed in the file system. mtr_buf_copy_t: A helper functor for copying a mini-transaction log. rb#6282 approved by Jimmy Yang

specific temporary errors The optimistic parallel slave's worker thread could face a run-time error due to the algorithm's specifics which allows for conflicts like the reported "Can't find record in 'table'". A typical stack is like {noformat} #0 handler::print_error (this=0x61c00008f8a0, error=149, errflag=0) at handler.cc:3650 #1 0x0000555555e95361 in write_record (thd=thd@entry=0x62a0000a2208, table=table@entry=0x61f00008ce88, info=info@entry=0x7fffdee356d0) at sql_insert.cc:1944 #2 0x0000555555ea7767 in mysql_insert (thd=thd@entry=0x62a0000a2208, table_list=0x61b00012ada0, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at sql_insert.cc:1039 #3 0x0000555555efda90 in mysql_execute_command (thd=thd@entry=0x62a0000a2208) at sql_parse.cc:3927 #4 0x0000555555f0cc50 in mysql_parse (thd=0x62a0000a2208, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at sql_parse.cc:7449 #5 0x00005555566d4444 in Query_log_event::do_apply_event (this=0x61200005b9c8, rgi=<optimized out>, query_arg=<optimized out>, q_len_arg=<optimized out>) at log_event.cc:4508 #6 0x00005555566d639e in Query_log_event::do_apply_event (this=<optimized out>, rgi=<optimized out>) at log_event.cc:4185 #7 0x0000555555d738cf in Log_event::apply_event (rgi=0x61d0001ea080, this=0x61200005b9c8) at log_event.h:1343 #8 apply_event_and_update_pos_apply (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080, reason=<optimized out>) at slave.cc:3479 #9 0x0000555555d8596b in apply_event_and_update_pos_for_parallel (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080) at slave.cc:3623 #10 0x00005555562aca83 in rpt_handle_event (qev=qev@entry=0x6190000fa088, rpt=rpt@entry=0x62200002bd68) at rpl_parallel.cc:50 #11 0x00005555562bd04e in handle_rpl_parallel_thread (arg=arg@entry=0x62200002bd68) at rpl_parallel.cc:1258 {noformat} Here {{handler::print_error}} computes whether to error log the current error when --log-warnings > 1. The decision flag is consulted bu {{my_message_sql()}} which can be eventually called. In the bug case the decision is to log. However in the optimistic mode slave applier case any conflict is attempted to resolve with rollback and retry to success. Hence the logging is at least extraneous. The case is fixed with adding a new flag {{ME_LOG_AS_WARN}} which {{handler::print_error}} may propagate further on through {{my_error}} when the error comes from an optimistically running slave worker thread. The new flag effectively requests the warning level for the errlog record, while the thread's DA records the actual error (which is regarded as temporary one by the parallel slave error handler).

This problem is similar to MDEV-10306. 1. Fixing Item_str_conv::val_str(String *str) to return the result in "str", and to use tmp_value only as a temporary buffer for args[0]->val_str(). The new code version now guarantees that the result is always returned in "str". The trick with copy_if_not_alloced() is not used any more. 2. The change #1 revealed the same problem in SUBSTRING_INDEX(), so some tests with combinations of UPPER()/LOWER() and SUBSTRING_INDEX() started to fail. Fixing Item_func_substr_index::val_str() the same way, to return the result in "str" and use tmp_value as a temporary buffer for args[0]->val_str().

…t operations with sequence The issue is that two MARIA_HA instances shares the same MARIA_STATUS_INFO object during UNION execution, so the second MARIA_HA instance state pointer MARIA_HA::state points to the MARIA_HA::state_save of the first MARIA instance. This happens in thr_multi_lock(...) { ... for (first_lock=data, pos= data+1 ; pos < end ; pos++) { ... if (pos[0]->lock == pos[-1]->lock && pos[0]->lock->copy_status) (pos[0]->lock->copy_status)((*pos)->status_param, (*first_lock)->status_param); ... } ... } Usually the state is restored from ha_maria::external_lock(...): \#0 _ma_update_status (param=0x6290000e6270) at ./storage/maria/ma_state.c:309 \#1 0x00005555577ccb15 in _ma_update_status_with_lock (info=0x6290000e6270) at ./storage/maria/ma_state.c:361 \#2 0x00005555577c7dcc in maria_lock_database (info=0x6290000e6270, lock_type=2) at ./storage/maria/ma_locking.c:66 \#3 0x0000555557802ccd in ha_maria::external_lock (this=0x61d0001b1308, thd=0x62a000048270, lock_type=2) at ./storage/maria/ha_maria.cc:2727 But _ma_update_status() does not take into account the case when MARIA_HA::status points to the MARIA_HA::state_save of the other MARIA_HA instance. The fix is to restore MARIA_HA::state in ha_maria::external_lock() after maria_lock_database() call for transactional tables.

Problem: ======== The mysqlbinlog tool is leaking memory, causing failures in various tests when compiling and testing with AddressSanitizer or LeakSanitizer like this: cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN:BOOL=ON /path/to/source make -j$(nproc) cd mysql-test ASAN_OPTIONS=abort_on_error=1 ./mtr --parallel=auto rpl.rpl_row_mysqlbinlog CURRENT_TEST: rpl.rpl_row_mysqlbinlog Direct leak of 112 byte(s) in 1 object(s) allocated from: #0 0x4eff87 in __interceptor_malloc (/dev/shm/5.5/client/mysqlbinlog+0x4eff87) #1 0x60eaab in my_malloc /mariadb/5.5/mysys/my_malloc.c:41:10 #2 0x5300dd in Log_event::read_log_event(char const*, unsigned int, char const**, Format_description_log_event const*, char) /mariadb/5.5/sql/log_event.cc:1568: #3 0x564a9c in dump_remote_log_entries(st_print_event_info*, char const*) /mariadb/5.5/client/mysqlbinlog.cc:1978:17 Analysis: ======== 'mysqlbinlog' tool is being used to read binary log events from a remote server. While reading binary log, if a fake rotate event is found following actions are taken. If 'to-last-log' option is specified, then fake rotate event is processed. In the absence of 'to-last-log' skip the fake rotate event. In this skipped case the fake rotate event object is not getting cleaned up resulting in memory leak. Fix: === Cleanup the fake rotate event. This issues is already fixed in MariaDB 10.0.23 and higher versions as part of commit c3018b0

Fix wrong typecast

There are 2 issues here: Issue #1: memory allocation. An IO_CACHE that uses encryption uses a larger buffer (it needs space for the encrypted data, decrypted data, IO_CACHE_CRYPT struct to describe encryption parameters etc). Issue #2: IO_CACHE::seek_not_done When IO_CACHE objects are cloned, they still share the file descriptor. This means, operation on one IO_CACHE may change the file read position which will confuse other IO_CACHEs using it. The fix of these issues would be: Allocate the buffer to also include the extra size needed for encryption. Perform seek again after one IO_CACHE reads the file.

Some DDL statements appear to acquire MDL locks for a table referenced by foreign key constraint from the actual affected table of the DDL statement. OPTIMIZE, REPAIR and ALTER TABLE belong to this class of DDL statements. Earlier MariaDB version did not take this in consideration, and appended only affected table in the certification key list in write set. Because of missing certification information, it could happen that e.g. OPTIMIZE table for FK child table could be allowed to apply in parallel with DML operating on the foreign key parent table, and this could lead to unhandled MDL lock conflicts between two high priority appliers (BF). The fix in this patch, changes the TOI replication for OPTIMIZE, REPAIR and ALTER TABLE statements so that before the execution of respective DDL statement, there is foreign key parent search round. This FK parent search contains following steps: * open and lock the affected table (with permissive shared locks) * iterate over foreign key contstraints and collect and array of Fk parent table names * close all tables open for the THD and release MDL locks * do the actual TOI replication with the affected table and FK parent table names as key values The patch contains also new mtr test for verifying that the above mentioned DDL statements replicate without problems when operating on FK child table. The mtr test scenario #1, which can be used to check if some other DDL (on top of OPTIMIZE, REPAIR and ALTER) could cause similar excessive FK parent table locking. Reviewed-by: Aleksey Midenkov <aleksey.midenkov@mariadb.com> Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

…warning in case it is executed in PS (prepared statement) mode The EXPLAIN EXTENDED statement run as a prepared statement can produce extra warning comparing with a case when EXPLAIN EXTENDED statement is run as a regular statement. For example, the following test case CREATE TABLE t1 (c int); CREATE TABLE t2 (d int); EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1; produces the extra warning "Field or reference 'c' of SELECT #2 was resolved in SELECT #1" in case the above mentioned "EXPLAIN EXTENDED" statement is executed in PS mode, that is by submitting the following statements: PREPARE stmt FROM "EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1"; EXECUTE stmt; The reason of the extra warning emittion is in a way items are handled (being fixed) during execution of the JOIN::prepare() method. The method Item_field::fix_fields() calls the find_field_in_tables() function in case a field hasn't been associated yet with the item. Implementation of the find_field_in_tables() function first checks whether a table containing the required field was already opened and cached. It is done by checking the data member item->cached_table. This data member is set on handling the PRERARE FROM statement and checked on executing the EXECUTE statement. If the data member item->cached_table is set the find_field_in_tables() function invoked and the mark_select_range_as_dependent() function called if the field is an outer referencee. The mark_select_range_as_dependent() function calls the mark_as_dependent() function that finally invokes the push_warning_printf() function that produces extra warning. To fix the issue, calling of push_warning_printf() is elimited in case it was run indirectly in result of hanlding already opened table from the Item_field::fix_fields() method.

Read of size 8 at 0x7fecf2e75fc8 by thread T2 (mutexes: write M1318): #0 tpool::thread_pool_generic::submit_task(tpool::task*) /tpool/tpool_generic.cc:823:9 (mariadbd+0x25fd2d2) #1 (anonymous namespace)::aio_uring::thread_routine((anonymous namespace)::aio_uring*) /tpool/aio_liburing.cc:173:20 (mariadbd+0x260b21b) #2 void std::__invoke_impl<void, void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(std::__invoke_other, void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x260c62a) #3 std::__invoke_result<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>::type std::__invoke<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x260c4ba) #4 void std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x260c442) #5 std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x260c3c5) #6 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x260c189) #7 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x7fecf2e75fc8 by main thread: #0 tpool::task::task(void (*)(void*), void*, tpool::task_group*) /tpool/task.cc:40:46 (mariadbd+0x260a138) #1 tpool::aiocb::aiocb() /tpool/tpool.h:147:13 (mariadbd+0x2355943) #2 void std::_Construct<tpool::aiocb>(tpool::aiocb*) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:109:38 (mariadbd+0x2355845) #3 tpool::aiocb* std::__uninitialized_default_n_1<false>::__uninit_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:579:3 (mariadbd+0x235576c) #4 tpool::aiocb* std::__uninitialized_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:638:14 (mariadbd+0x23556e9) #5 tpool::aiocb* std::__uninitialized_default_n_a<tpool::aiocb*, unsigned long, tpool::aiocb>(tpool::aiocb*, unsigned long, std::allocator<tpool::aiocb>&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:704:14 (mariadbd+0x2355641) #6 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::_M_default_initialize(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1606:4 (mariadbd+0x2354f3d) #7 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::vector(unsigned long, std::allocator<tpool::aiocb> const&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:512:9 (mariadbd+0x2354a19) #8 tpool::cache<tpool::aiocb>::cache(unsigned long, tpool::cache_notification_mode) /tpool/tpool_structs.h:73:20 (mariadbd+0x2354784) #9 io_slots::io_slots(int, int) /storage/innobase/os/os0file.cc:93:3 (mariadbd+0x235343b) #10 os_aio_init() /storage/innobase/os/os0file.cc:3780:22 (mariadbd+0x234ebce) #11 srv_start(bool) /storage/innobase/srv/srv0start.cc:1190:6 (mariadbd+0x256720c) #12 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed3bda) #13 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7be06) #14 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fa1b) #15 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f07f) #16 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd70fb2) #17 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6a9d7) #18 main /sql/main.cc:34:10 (mariadbd+0xd65d18) I think the report is incorrect: it's not possible to have such a race condition. I've checked it by reading the code and putting assertions. Namely, no aio I/O is possible before the end of os_aio_init(). Most probably it's some bug in TSAN. But the patch fixes around 5 related reports and this is a step toward TSAN usefullness. Currently it reports too much noise. std::unique_ptr is a step toward https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#r11-avoid-calling-new-and-delete-explicitly There is no std::make_unique() in C++11, however.

WARNING: ThreadSanitizer: data race (pid=1503350) Write of size 8 at 0x0000067b1f20 by thread T3: #0 os_file_sync_posix(int) /storage/innobase/os/os0file.cc:895:5 (mariadbd+0x23493f6) #1 os_file_flush_func(int) /storage/innobase/os/os0file.cc:983:8 (mariadbd+0x2349204) #2 file_os_io::flush() /storage/innobase/log/log0log.cc:326:10 (mariadbd+0x22eaaa9) #3 log_file_t::flush() /storage/innobase/log/log0log.cc:440:18 (mariadbd+0x22eb2d0) #4 log_t::file::flush() /storage/innobase/log/log0log.cc:507:29 (mariadbd+0x22ebe69) #5 log_write_flush_to_disk_low(unsigned long) /storage/innobase/log/log0log.cc:629:17 (mariadbd+0x22ed3f3) #6 log_write_up_to(unsigned long, bool, bool, completion_callback const*) /storage/innobase/log/log0log.cc:829:3 (mariadbd+0x22ecb04) #7 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1734:5 (mariadbd+0x20d37f1) #8 buf_flush_sync_for_checkpoint(unsigned long) /storage/innobase/buf/buf0flu.cc:1947:7 (mariadbd+0x20d4193) #9 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2186:9 (mariadbd+0x20cdad7) #10 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3aaa) #11 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c39bd) #12 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3965) #13 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c3905) #14 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37f9) #15 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x0000067b1f20 by main thread: #0 os_file_sync_posix(int) /storage/innobase/os/os0file.cc:895:5 (mariadbd+0x23493f6) #1 os_file_flush_func(int) /storage/innobase/os/os0file.cc:983:8 (mariadbd+0x2349204) #2 fil_space_t::flush_low() /storage/innobase/fil/fil0fil.cc:504:5 (mariadbd+0x205cad5) #3 fil_flush_file_spaces() /storage/innobase/fil/fil0fil.cc:2947:13 (mariadbd+0x206523f) #4 log_checkpoint() /storage/innobase/buf/buf0flu.cc:1777:5 (mariadbd+0x20cd069) #5 buf_flush_wait_flushed(unsigned long) /storage/innobase/buf/buf0flu.cc:1867:5 (mariadbd+0x20ccf95) #6 log_make_checkpoint() /storage/innobase/buf/buf0flu.cc:1793:3 (mariadbd+0x20cc4c9) #7 buf_dblwr_t::create() /storage/innobase/buf/buf0dblwr.cc:216:3 (mariadbd+0x209076a) #8 srv_start(bool) /storage/innobase/srv/srv0start.cc:1685:20 (mariadbd+0x256b514) #9 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed406a) #10 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7c246) #11 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fe6b) #12 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f4cf) #13 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd713f2) #14 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6ae17) #15 main /sql/main.cc:34:10 (mariadbd+0xd66158) This is a correct report by TSAN for an obvious case: unprotected global counter. Fix it by making counter std::atomic.

WARNING: ThreadSanitizer: data race (pid=1506937) Write of size 8 at 0x0000067ab740 by thread T6: #0 buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:2946:8 (mariadbd+0x2014c7f) #1 buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:3047:10 (mariadbd+0x2016216) #2 btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, ssux_lock_impl<true>*, mtr_t*, unsigned long) /storage/innobase/btr/btr0cur.cc:1613:10 (mariadbd+0x1fb5bff) #3 btr_pcur_open_low(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_pcur_t*, unsigned long, mtr_t*) /storage/innobase/include/btr0pcur.ic:439:8 (mariadbd+0x24ddead) #4 row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t const*, dtuple_t const*, mtr_t*) /storage/innobase/row/row0row.cc:1215:7 (mariadbd+0x24dd537) #5 row_purge_reposition_pcur(unsigned long, purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:81:23 (mariadbd+0x24c5369) #6 row_purge_reset_trx_id(purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:748:6 (mariadbd+0x24c90c7) #7 row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool) /storage/innobase/row/row0purge.cc:1174:4 (mariadbd+0x24c8262) #8 row_purge(purge_node_t*, unsigned char*, que_thr_t*) /storage/innobase/row/row0purge.cc:1218:18 (mariadbd+0x24c5af3) #9 row_purge_step(que_thr_t*) /storage/innobase/row/row0purge.cc:1267:3 (mariadbd+0x24c5996) #10 que_thr_step(que_thr_t*) /storage/innobase/que/que0que.cc:653:9 (mariadbd+0x23d5298) #11 que_run_threads_low(que_thr_t*) /storage/innobase/que/que0que.cc:709:25 (mariadbd+0x23d3f29) #12 que_run_threads(que_thr_t*) /storage/innobase/que/que0que.cc:729:2 (mariadbd+0x23d3bdf) #13 srv_task_execute() /storage/innobase/srv/srv0srv.cc:1692:3 (mariadbd+0x2562841) #14 purge_worker_callback(void*) /storage/innobase/srv/srv0srv.cc:1864:10 (mariadbd+0x255f361) #15 tpool::task_group::execute(tpool::task*) /tpool/task_group.cc:55:9 (mariadbd+0x260a5ca) #16 tpool::task::execute() /tpool/task.cc:47:16 (mariadbd+0x260adf6) #17 tpool::thread_pool_generic::worker_main(tpool::worker_data*) /tpool/tpool_generic.cc:550:11 (mariadbd+0x25fc590) #18 void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14 (mariadbd+0x26061b5) #19 std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x2605f57) #20 void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x2605ecb) #21 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x2605e35) #22 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x2605ac9) #23 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x0000067ab740 by thread T8: #0 buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:2946:8 (mariadbd+0x2014c7f) #1 buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:3047:10 (mariadbd+0x2016216) #2 btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, ssux_lock_impl<true>*, mtr_t*, unsigned long) /storage/innobase/btr/btr0cur.cc:1613:10 (mariadbd+0x1fb5bff) #3 btr_pcur_open_low(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_pcur_t*, unsigned long, mtr_t*) /storage/innobase/include/btr0pcur.ic:439:8 (mariadbd+0x24ddead) #4 row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t const*, dtuple_t const*, mtr_t*) /storage/innobase/row/row0row.cc:1215:7 (mariadbd+0x24dd537) #5 row_purge_reposition_pcur(unsigned long, purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:81:23 (mariadbd+0x24c5369) #6 row_purge_reset_trx_id(purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:748:6 (mariadbd+0x24c90c7) #7 row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool) /storage/innobase/row/row0purge.cc:1174:4 (mariadbd+0x24c8262) #8 row_purge(purge_node_t*, unsigned char*, que_thr_t*) /storage/innobase/row/row0purge.cc:1218:18 (mariadbd+0x24c5af3) #9 row_purge_step(que_thr_t*) /storage/innobase/row/row0purge.cc:1267:3 (mariadbd+0x24c5996) #10 que_thr_step(que_thr_t*) /storage/innobase/que/que0que.cc:653:9 (mariadbd+0x23d5298) #11 que_run_threads_low(que_thr_t*) /storage/innobase/que/que0que.cc:709:25 (mariadbd+0x23d3f29) #12 que_run_threads(que_thr_t*) /storage/innobase/que/que0que.cc:729:2 (mariadbd+0x23d3bdf) #13 trx_purge(unsigned long, bool) /storage/innobase/trx/trx0purge.cc:1271:2 (mariadbd+0x25841b4) #14 srv_do_purge(unsigned long*) /storage/innobase/srv/srv0srv.cc:1784:20 (mariadbd+0x2563224) #15 purge_coordinator_callback_low() /storage/innobase/srv/srv0srv.cc:1881:35 (mariadbd+0x2562b3b) #16 purge_coordinator_callback(void*) /storage/innobase/srv/srv0srv.cc:1910:3 (mariadbd+0x255f4ab) #17 tpool::task_group::execute(tpool::task*) /tpool/task_group.cc:55:9 (mariadbd+0x260a5ca) #18 tpool::task::execute() /tpool/task.cc:47:16 (mariadbd+0x260adf6) #19 tpool::thread_pool_generic::worker_main(tpool::worker_data*) /tpool/tpool_generic.cc:550:11 (mariadbd+0x25fc590) #20 void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14 (mariadbd+0x26061b5) #21 std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x2605f57) #22 void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x2605ecb) #23 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x2605e35) #24 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x2605ac9) #25 <null> <null> (libstdc++.so.6+0xd230f) Location is global 'buf_dbg_counter' of size 8 at 0x0000067ab740 (mariadbd+0x67ab740) The obvious fix is to make counter atomic.

Write of size 1 at 0x0000067abe08 by thread T3 (mutexes: write M1372): #0 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2366:29 (mariadbd+0x20cea7c) #1 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3a8a) #2 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c399d) #3 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3945) #4 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c38e5) #5 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37d9) #6 <null> <null> (libstdc++.so.6+0xd230f) Previous read of size 1 at 0x0000067abe08 by main thread: #0 logs_empty_and_mark_files_at_shutdown() /storage/innobase/log/log0log.cc:1094:6 (mariadbd+0x22eeff3) #1 innodb_shutdown() /storage/innobase/srv/srv0start.cc:1970:3 (mariadbd+0x256ffd6) #2 innobase_end(handlerton*, ha_panic_function) /storage/innobase/handler/ha_innodb.cc:4265:3 (mariadbd+0x1ee3fc4) #3 ha_finalize_handlerton(st_plugin_int*) /sql/handler.cc:595:5 (mariadbd+0xf7bac9) #4 plugin_deinitialize(st_plugin_int*, bool) /sql/sql_plugin.cc:1266:9 (mariadbd+0x1611789) #5 reap_plugins() /sql/sql_plugin.cc:1342:7 (mariadbd+0x160e17d) #6 plugin_shutdown() /sql/sql_plugin.cc:2050:7 (mariadbd+0x1611f42) #7 clean_up(bool) /sql/mysqld.cc:1923:3 (mariadbd+0xd67a4c) #8 unireg_abort /sql/mysqld.cc:1835:3 (mariadbd+0xd67605) #9 mysqld_main(int, char**) /sql/mysqld.cc:5741:7 (mariadbd+0xd6b36a) #10 main /sql/main.cc:34:10 (mariadbd+0xd661a8) Location is global 'buf_page_cleaner_is_active' of size 1 at 0x0000067abe08 (mariadbd+0x67abe08)

WARNING: ThreadSanitizer: data race (pid=1510842) Write of size 8 at 0x0000067b1e98 by main thread: #0 os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /storage/innobase/os/os0file.cc:2928:2 (mariadbd+0x234c5ac) #1 os_file_write_func(IORequest const&, char const*, int, void const*, unsigned long, unsigned long) /storage/innobase/os/os0file.cc:2963:20 (mariadbd+0x234c019) #2 file_os_io::write(char const*, unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:320:10 (mariadbd+0x22eaa50) #3 log_file_t::write(unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:434:18 (mariadbd+0x22eb1d8) #4 log_t::file::write(unsigned long, st_::span<unsigned char>) /storage/innobase/log/log0log.cc:496:29 (mariadbd+0x22ebb55) #5 log_write_buf(unsigned char*, unsigned long, unsigned long, unsigned long, unsigned long) /storage/innobase/log/log0log.cc:614:14 (mariadbd+0x22f1b51) #6 log_write(bool) /storage/innobase/log/log0log.cc:755:2 (mariadbd+0x22ed2ec) #7 log_write_up_to(unsigned long, bool, bool, completion_callback const*) /storage/innobase/log/log0log.cc:817:5 (mariadbd+0x22eca44) #8 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1734:5 (mariadbd+0x20d37c1) #9 log_checkpoint() /storage/innobase/buf/buf0flu.cc:1787:10 (mariadbd+0x20cd155) #10 buf_flush_wait_flushed(unsigned long) /storage/innobase/buf/buf0flu.cc:1867:5 (mariadbd+0x20ccf8f) #11 log_make_checkpoint() /storage/innobase/buf/buf0flu.cc:1793:3 (mariadbd+0x20cc4c9) #12 buf_dblwr_t::create() /storage/innobase/buf/buf0dblwr.cc:216:3 (mariadbd+0x209076a) #13 srv_start(bool) /storage/innobase/srv/srv0start.cc:1685:20 (mariadbd+0x256b4aa) #14 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed40da) #15 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7c2b6) #16 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fedb) #17 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f53f) #18 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd71462) #19 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6ae87) #20 main /sql/main.cc:34:10 (mariadbd+0xd661c8) Previous write of size 8 at 0x0000067b1e98 by thread T3: #0 os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /storage/innobase/os/os0file.cc:2928:2 (mariadbd+0x234c5ac) #1 os_file_write_func(IORequest const&, char const*, int, void const*, unsigned long, unsigned long) /storage/innobase/os/os0file.cc:2963:20 (mariadbd+0x234c019) #2 file_os_io::write(char const*, unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:320:10 (mariadbd+0x22eaa50) #3 log_file_t::write(unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:434:18 (mariadbd+0x22eb1d8) #4 log_t::file::write(unsigned long, st_::span<unsigned char>) /storage/innobase/log/log0log.cc:496:29 (mariadbd+0x22ebb55) #5 log_write_checkpoint_info(unsigned long) /storage/innobase/log/log0log.cc:911:14 (mariadbd+0x22edd4e) #6 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1755:3 (mariadbd+0x20d3a3d) #7 buf_flush_sync_for_checkpoint(unsigned long) /storage/innobase/buf/buf0flu.cc:1947:7 (mariadbd+0x20d4163) #8 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2186:9 (mariadbd+0x20cdab1) #9 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3aaa) #10 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c39bd) #11 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3965) #12 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c3905) #13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37f9) #14 <null> <null> (libstdc++.so.6+0xd230f) Location is global 'os_n_file_writes' of size 8 at 0x0000067b1e98 (mariadbd+0x67b1e98) Make variable atomic.

In commit 1193a79 we set innodb_use_native_aio=OFF when using io_uring on a kernel where write requests could potentially be lost. The last reproducible issue was fixed in Linux 5.16-rc1 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.16-rc1&id=d3e3c102d107bb84251455a298cf475f24bab995 and the fix was backported to 5.15.3. Hence, using a 5.16 or later kernel should be fine. The Debian kernel 5.15.0-1-amd64 (5.15.3-1) was tested. On Debian, utsname::release or uname -r does not reflect the exact minor version while utsname::version and uname -v does. On Fedora however the utsname::version is rather different: $ uname -r 5.14.20-200.fc34.x86_64 $ uname -v #1 SMP Thu Nov 18 22:03:20 UTC 2021 As such we use the version, but fall back to the release if there isn't the beginnings of a kernel version in the version. Thanks to Daniel Black for reporting the Linux kernel bug and Jens Axboe for actually fixing it. Co-Authored-By: Daniel Black <daniel@mariadb.org> Closes: MariaDB#1953

…nd execution of PS Consider the following use case: MariaDB [test]> CREATE TABLE t1 (field1 BIGINT DEFAULT -1); MariaDB [test]> CREATE VIEW v1 AS SELECT DISTINCT field1 FROM t1; Repeated execution of the following query as a Prepared Statement MariaDB [test]> PREPARE stmt FROM 'SELECT * FROM v1 WHERE field1 <=> NULL'; MariaDB [test]> EXECUTE stmt; results in a crash for a server built with DEBUG. MariaDB [test]> EXECUTE stmt; ERROR 2013 (HY000): Lost connection to MySQL server during query Assertion failed: (!result), function convert_const_to_int, file item_cmpfunc.cc, line 476. Abort trap: 6 (core dumped) The crash inside the function convert_const_to_int() happens by the reason that the value -1 is stored in an instance of the class Field_longlong on restoring its original value in the statement result= field->store(orig_field_val, TRUE); that leads to assigning the value 1 to the variable 'result' with subsequent crash in the DBUG_ASSERT statement following it DBUG_ASSERT(!result); The main matter here is why this assertion failure happens on the second execution of the prepared statement and doens't on the first one. On first handling of the statement 'EXECUTE stmt;' a temporary table is created for serving the query involving the view 'v1'. The table is created by the function create_tmp_table() in the following calls trace: (trace #1) JOIN::prepare (at sql_select.cc:725) st_select_lex::handle_derived LEX::handle_list_of_derived TABLE_LIST::handle_derived mysql_handle_single_derived mysql_derived_prepare select_union::create_result_table create_tmp_table Note, that the data member TABLE::status of a TABLE instance returned by the function create_tmp_table() has the value 0. Later the function setup_table_map() is called on the TABLE instance just created for the sake of the temporary table (calls trace #2 is below): JOIN::prepare (at sql_select.cc:737) setup_tables_and_check_access setup_tables setup_table_map where the data member TABLE::status is set to the value STATUS_NO_RECORD. After that when execution of the method JOIN::prepare reaches calling of the function setup_without_group() the following calls trace is invoked JOIN::prepare setup_without_group setup_conds Item_func::fix_fields Item_func_equal::fix_length_and_dec Item_bool_rowready_func2::fix_length_and_dec Item_func::setup_args_and_comparator Item_func::convert_const_compared_to_int_field convert_const_to_int There is the following code snippet in the function convert_const_to_int() at the line item_cmpfunc.cc:448 bool save_field_value= (field_item->const_item() || !(field->table->status & STATUS_NO_RECORD)); Since field->table->status has bits STATUS_NO_RECORD set the variable save_field_value is false and therefore neither the method Field_longlong::val_int() nor the method Field_longlong::store is called on the Field instance that has the numeric value -1. That is the reason why first execution of the Prepared Statement for the query 'SELECT * FROM v1 WHERE field1 <=> NULL' is successful. On second running of the statement 'EXECUTE stmt' a new temporary tables is also created by running the calls trace #1 but the trace #2 is not executed by the reason that data member SELECT_LEX::first_cond_optimization has been set to false on first execution of the prepared statemet (in the method JOIN::optimize_inner()). As a consequence, the data member TABLE::status for a temporary table just created doesn't have the flags STATUS_NO_RECORD set and therefore on re-execution of the prepared statement the methods Field_longlong::val_int() and Field_longlong::store() are called for the field having the value -1 and the DBUG_ASSERT(!result) is fired. To fix the issue the data member TABLE::status has to be assigned the value STATUS_NO_RECORD in every place where the macros empty_record() is called to emptify a record for just instantiated TABLE object created on behalf the new temporary table.

Basic ideas: 1. Store "popular" values in their own buckets. 2. Also store ndv (Number of Distinct Values) in each bucket. Because of #1, the buckets are now variable-size, so store the size in each bucket. Adjust selectivity estimation functions accordingly.

…iers for stored functions Changes: 1. Enabling IN/OUT/INOUT mode for sql_mode=DEFAULT, adding tests for sql_mode=DEFAULT based by mostly translating compat/oracle.sp-inout.test to SQL/PSM with minor changes (e.g. testing trigger OLD.column and NEW.column as IN/OUT parameters). 2. Removing duplicate grammar: sp_pdparam and sp_fdparam implemented exactly the same syntax after - the first patch for MDEV-10654 (for sql_mode=ORACLE) - the change #1 from this patch (for sql_mode=DEFAULT) Removing separate rules and adding a single "sp_param" rule instead, which now covers both PRDEDURE and FUNCTION parameters (and CURSOR parameters as well!). 3. Adding a helper rule sp_param_name_and_mode, which is a combination of the parameter name and the IN/OUT/INOUT mode. It allows to simplify the grammer a bit. 4. The first patch unintentionally allowed IN/OUT/INOUT mode to be specified in CURSOR parameters. This is good for the IN keyword - it is allowed in PL/SQL CURSORs. This is not good the the OUT/INOUT keywords - they should not be allowed. Adding a additional symantic post-check.

…and UNION (Edits by SergeiP: fix encryption.tempfiles_encrypted, re-word comment) Global ORDER BY clause of a UNION may not refer to 1) aggregate functions or 2) window functions. setup_order() checked for #1 but not for #2.

Recent adventures in liburing and btrfs have shown up some kernel version dependent bugs. Having a bug report of accurace kernel version can start to correlate these errors sooner. On Linux, /proc/version contains the kernel version. FreeBSD has kern.version (per man 8 sysctl), so include that too. Example output: Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h Kernel version: Linux version 5.19.0-0.rc2.21.fc37.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1), GNU ld version 2.38-14.fc37) #1 SMP PREEMPT_DYNAMIC Mon Jun 13 15:27:24 UTC 2022 Segmentation fault (core dumped)

Adding two levels of optimization: 1. For every bytes pair [00..FF][00..FF] which: a. consists of two ASCII characters or makes a well-formed two-byte character b. whose total weight string fits into 4 weights (concatenated weight string in case of two ASCII characters, or a single weight string in case of a two-byte character) c. whose weight is context independent (i.e. does not depend on contractions or previous context pairs) store weights in a separate array of MY_UCA_2BYTES_ITEM, so during scanner_next() we can scan two bytes at a time. Byte pairs that do not match the conditions a-c are marked in this array as not applicable for optimization and scanned as before. 2. For every byte pair which is applicable for optimization in #1, and which produces only one or two weights, store weights in one more array of MY_UCA_WEIGHT2. So in the beginning of strnncoll*() we can skip equal prefixes using an even more efficient loop. This loop consumes two bytes at a time. The loop scans while the two bytes on both sides produce weight strings of equal length (i.e. one weight on both sides, or two weight on both sides). This allows to compare efficiently: - Context independent sequences consisting of two ASCII characters - Context independent 2-byte characters - Contractions consisting of two ASCII characters, e.g. Czech "ch". - Some tricky cases: "ss" vs "SHARP S" ("ss" produces two weights, 0xC39F also produces two weights)

(Based on original patch by Oleksandr Byelkin) Multi-table DELETE can execute via "buffered" mode: at phase #1 it collects rowids of rows to be deleted, then at phase #2 in multi_delete::do_deletes() it calls handler->rnd_pos() to read rows to be deleted and deletes them. The problem occurred when phase #1 used Rowid Filter on the table that phase #2 would be deleting from. In InnoDB, h->rnd_init(scan=false) and h->rnd_pos() is an index scan over PK under the hood. So, at phase #2 ha_innobase::rnd_init() would try to use the Rowid Filter and hit an assertion inside ha_innobase::rnd_init(). Note that multi-table UPDATE works similarly but was not affected, because patch for MDEV-7487 added code to disable rowid filter for phase #2 in multi_update::do_updates(). This patch changes the approach: - It makes InnoDB not use Rowid Filter in rnd_pos() scans: it is disabled in ha_innobase::rnd_init() and enabled back in ha_innobase::rnd_end(). - multi_update::do_updates() no longer disables Rowid Filter for phase#2 as it is no longer necessary.

(Based on the original patch by Jason Cu) Part #1: - Add ha_handler_stats::{icp_attempts,icp_match}, make handler_index_cond_check() increment them. - ANALYZE FORMAT=JSON now prints r_icp_filtered based on these counters.

Fixing applying the COLLATE clause to a parameter caused an error error: COLLATION '...' is not valid for CHARACTER SET 'binary' Fix: - Changing the collation derivation for a non-prepared Item_param to DERIVATION_IGNORABLE. - Allowing to apply any COLLATE clause to expressions with DERIVATION_IGNORABLE. This includes: 1. A non-prepared Item_param 2. An explicit NULL 3. Expressions derived from #1 and #2 For example: SELECT ? COLLATE utf8mb_unicode_ci; SELECT NULL COLLATE utf8mb_unicode_ci; SELECT CONCAT(?) COLLATE utf8mb_unicode_ci; SELECT CONCAT(NULL) COLLATE utf8mb_unicode_ci - Additional change: preserving the collation of an expression when the expression gets assigned to a PS parameter and evaluates to SQL NULL. Before this change, the collation of the parameter was erroneously set to &my_charset_binary. - Additional change: removing the multiplication to mbmaxlen from the fix_char_length_ulonglong() argument, because the multiplication already happens inside fix_char_length_ulonglong(). This fixes a too large column size created for a COLLATE clause.

here MSAN complains that ==218853==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7f84a77c60a3 in _gnutls_rnd_init /tmp/msan/lib/random.c:69:6 #1 0x7f84a77c60a3 in gnutls_rnd /tmp/msan/lib/random.c:168:6 but the line lib/random.c:69 in gnutls-3.7.1 is 69 if (unlikely(!rnd_initialized)) { and rnd_initialized is declared as 40 static _Thread_local unsigned rnd_initialized = 0; which apparently MSAN isn't happy with

midenok added the Interesting color label Oct 5, 2016

midenok pushed a commit that referenced this issue Jun 12, 2019

MyRocks: Post-merge testcase fixes part #1

a01823a

midenok mentioned this issue Aug 19, 2019

Pruning bugs #58

Open

midenok mentioned this issue Oct 14, 2019

MDEV-16210 FK constraints on versioned tables use historical rows, which may cause constraint violation #60

Closed

midenok mentioned this issue Oct 21, 2019

Partitioning features #53

Open

midenok pushed a commit that referenced this issue Jan 4, 2020

MDEV-21341: Fix optimizer-related UBSAN failures, part #1:

0ac0bc8

Fix wrong typecast

midenok mentioned this issue Mar 23, 2020

MDEV-20515 Table corruption or Assertion `bitmap_is_set(&(m_part_info->read_partitions), part_id)' failed upon multi-update on a versioned partitioned table #71

Closed

midenok pushed a commit that referenced this issue Mar 22, 2022

Code cleanup part #1

72c0ba4

midenok mentioned this issue Jan 25, 2023

SAMU-64 Allow administrators to enable or disable parallel replication on a per-table basis #109

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Interesting label colors #1

Interesting label colors #1

midenok commented Oct 5, 2016

Interesting label colors #1

Interesting label colors #1

Comments

midenok commented Oct 5, 2016