Bug#32416811: Handler.ha_rnd_next failure

This failure may occur when we have a cursor created inside a
stored procedure and need to recreate the temporary table behind
the cursor because the storage limitation of the primary temporary
table is exceeded. In that case, the code that populates the temporary
table will recreate the temporary table in another storage engine, move
the contents of the primary table to the new one, and insert the
remaining part of the query result to this new table.

However, the handler for the new table is created on the current
THD's mem_root, which may not provide the storage lifetime expected
by cursors. Cursor objects exist over several roundtrips and we need
to enforce proper lifetime for them. For the primary temporay table,
this is ensured by using a special mem_root created in the cursor object.
However, we have no control over when the new temporary table may be
created, thus we may use the wrong mem_root for the allocation.

The fix is to save the memory allocator used when instantiating the
primary temporary table, and ensure to use the same allocator when
recreating the table with a different storage engine.
The pointer to the memory allocator is stored in the TABLE_SHARE object
and is only used for internal temporary tables.

Reviewed by: Sergey Glukhov <sergey.glukhov@oracle.com>

Author: roylyseng

sql/sql_tmp_table.cc

@@ -2081,7 +2081,10 @@ bool setup_tmp_table_handler(THD *thd, TABLE *table, ulonglong select_options,
   }
   assert(share->db_plugin != nullptr);
 
-  table->file = get_new_handler(share, false, thd->mem_root, share->db_type());
+  share->alloc_for_tmp_file_handler = thd->mem_root;
+
+  table->file = get_new_handler(share, false, share->alloc_for_tmp_file_handler,
+                                share->db_type());
   if (table->file == nullptr) return true;
 
   // Update the handler with information about the table object
@@ -2225,7 +2228,8 @@ static bool create_tmp_table_with_fallback(THD *thd, TABLE *table) {
       table->file->create(share->table_name.str, table, &create_info, nullptr);
   if (error == HA_ERR_RECORD_FILE_FULL &&
       table->s->db_type() == temptable_hton) {
-    table->file = get_new_handler(table->s, false, thd->mem_root, innodb_hton);
+    table->file = get_new_handler(
+        table->s, false, share->alloc_for_tmp_file_handler, innodb_hton);
     error = table->file->create(share->table_name.str, table, &create_info,
                                 nullptr);
   }
@@ -2580,8 +2584,10 @@ bool create_ondisk_from_heap(THD *thd, TABLE *wtable, int error,
 
     new_table.s = &share;  // New table points to new share
 
-    if (!(new_table.file = get_new_handler(&share, false, thd->mem_root,
-                                           new_table.s->db_type())))
+    new_table.file =
+        get_new_handler(&share, false, old_share->alloc_for_tmp_file_handler,
+                        new_table.s->db_type());
+    if (new_table.file == nullptr)
       goto err_after_proc_info; /* purecov: inspected */
     if (new_table.file->set_ha_share_ref(&share.ha_share))
       goto err_after_alloc; /* purecov: inspected */

sql/table.h

@@ -707,6 +707,12 @@ struct TABLE_SHARE {
   /* hash of field names (contains pointers to elements of field array) */
   collation_unordered_map<std::string, Field **> *name_hash{nullptr};
   MEM_ROOT mem_root;
+  /**
+    Used to allocate new handler for internal temporary table when the
+    size limitation of the primary storage engine is exceeded.
+  */
+  MEM_ROOT *alloc_for_tmp_file_handler{nullptr};
+
   TYPELIB keynames;            /* Pointers to keynames */
   TYPELIB *intervals{nullptr}; /* pointer to interval info */
   mysql_mutex_t LOCK_ha_data;  /* To protect access to ha_data */