MDEV-22890 DEADLOCK of threads detected: row0sel.cc S-LOCK / btr0cur.…

…cc S-LOCK / row0quiesce.cc X-LOCK

Problem:
=======
- Read operations are always allowed to hold a secondary index leaf
latch and then look up the corresponding clustered index record.
Flush table operation acquires secondary index latch while holding
a clustered index latch. It leads to deadlock violation.

Fix:
====
- Flush table operation should acquire secondary index before taking
clustered index to avoid deadlock violation with select operation.

storage/innobase/include/dict0dict.ic

@@ -1026,16 +1026,18 @@ dict_table_x_lock_indexes(
 /*======================*/
 	dict_table_t*	table)	/*!< in: table */
 {
-	dict_index_t*   index;
-
 	ut_ad(mutex_own(&dict_sys->mutex));
 
+	dict_index_t* clust_index = dict_table_get_first_index(table);
+
 	/* Loop through each index of the table and lock them */
-	for (index = dict_table_get_first_index(table);
+	for (dict_index_t* index = dict_table_get_next_index(clust_index);
 	     index != NULL;
 	     index = dict_table_get_next_index(index)) {
 		rw_lock_x_lock(dict_index_get_lock(index));
 	}
+
+	rw_lock_x_lock(dict_index_get_lock(clust_index));
 }
 
 /*********************************************************************//**

storage/xtradb/include/dict0dict.ic

@@ -1026,16 +1026,18 @@ dict_table_x_lock_indexes(
 /*======================*/
 	dict_table_t*	table)	/*!< in: table */
 {
-	dict_index_t*   index;
-
 	ut_ad(mutex_own(&dict_sys->mutex));
 
+	dict_index_t* clust_index = dict_table_get_first_index(table);
+
 	/* Loop through each index of the table and lock them */
-	for (index = dict_table_get_first_index(table);
+	for (dict_index_t* index = dict_table_get_next_index(clust_index);
 	     index != NULL;
 	     index = dict_table_get_next_index(index)) {
 		rw_lock_x_lock(dict_index_get_lock(index));
 	}
+
+	rw_lock_x_lock(dict_index_get_lock(clust_index));
 }
 
 /*********************************************************************//**