Bug #28772251 "SWAPPING COLUMN BY ALTER TABLE RENAME COLUMN BREAKS STORED GENERATED COLUMN".

Stored generated column values and indexes on virtual generated columns
were not correctly updated after columns on which these generated columns
depended were swapped using ALTER TABLE ... RENAME COLUMN or ALTER TABLE
... CHANGE COLUMN.

This problem was caused by the fact that renaming of columns on which
generated columns were dependent was not prohibited as it was planned
initially (we only detected the problem if generation expression became
invalid). Nor the fact that base columns were swapped and generated
expression needs re-evaluation as result was detected.

This fix solves the problem by disallowing renaming of base columns
for generated columns, generated defaults and functional indexes.

However, such renaming is allowed if generated column/default or
functional index is removed by the same ALTER TABLE. We also allow
such renaming if dependent expression in question is updated within
the same ALTER TABLE. This allows to support existing/documented user
scenarios in which generation expression was updated to follow base
column renames.

Restriction on dropping columns on which generated columns, defaults
or functional indexes depend was relaxed in similar way.

Code which evaluates flags for in-place/instant ALTER TABLE SE API
was adjusted to detect situations in which generated column expressions
stays the same but has changed its values due to base column
swapping or replacement (through drop/add).

To support the above changes handling of Alter_info::drop_list list
by SQL-layer code has been changed. We no longer delete elements
which correspond to columns and keys from it. As consequence InnoDB
SE code has been adjusted to take this change into account. To support
the latter new members for number of added and dropped virtual columns
were added to Alter_inplace_info class.

New error message and code was added to report about dependent
functional indexes for columns being renamed/dropped. Existing
error message about dependent default expressions was adjusted to
make it possible to use in cases when base columns are renamed.

Reviewed-by: Sivert Sorumgaard <sivert.sorumgaard@oracle.com>
Reviewed-by: Bin Su <bin.x.su@oracle.com>

Author: Dmitry Lenev

mysql-test/r/alter_table.result

@@ -3487,9 +3487,9 @@ t_gen	CREATE TABLE `t_gen` (
   `b` double GENERATED ALWAYS AS (sqrt(`c`)) VIRTUAL
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
 ALTER TABLE t_gen CHANGE COLUMN c a INT;
-ERROR 42S22: Unknown column 'c' in 'generated column function'
+ERROR HY000: Column 'c' has a generated column dependency.
 ALTER TABLE t_gen RENAME COLUMN c TO a;
-ERROR 42S22: Unknown column 'c' in 'generated column function'
+ERROR HY000: Column 'c' has a generated column dependency.
 DROP TABLE t_gen;
 CREATE TABLE foo (col1 INT);
 INSERT INTO foo VALUES (1), (2);

mysql-test/r/default_as_expr.result

@@ -240,7 +240,7 @@ ERROR HY000: DEFAULT function cannot be used with default value expressions
 SELECT DEFAULT(c) from t3;
 ERROR HY000: DEFAULT function cannot be used with default value expressions
 ALTER TABLE t3 DROP COLUMN d;
-ERROR HY000: Column 'd' of table 't3' has a default value expression dependency and cannot be dropped.
+ERROR HY000: Column 'd' of table 't3' has a default value expression dependency and cannot be dropped or renamed.
 DROP TABLE t3;
 #
 # Test UTF8
@@ -934,14 +934,12 @@ DROP TABLE t2;
 # BUG#28040739 - WL#9418: INCORRECT ERROR THROWN WITH RENAME COLUMN
 #
 CREATE TABLE t1 ( i INT, j INT DEFAULT( i * i) ) ;
-# "Unknown column 'i' in 'default value expression'"
 ALTER TABLE t1 RENAME COLUMN i to i1 ;
-ERROR 42S22: Unknown column 'i' in 'default value expression'
+ERROR HY000: Column 'i' of table 't1' has a default value expression dependency and cannot be dropped or renamed.
 DROP TABLE t1;
 CREATE TABLE t1 ( i INT, j INT DEFAULT (i * i) ) ;
-# "Unknown column 'i' in 'default value expression'"
 ALTER TABLE t1 CHANGE COLUMN i i1 DOUBLE DEFAULT ( 4 * 4 ) ;
-ERROR 42S22: Unknown column 'i' in 'default value expression'
+ERROR HY000: Column 'i' of table 't1' has a default value expression dependency and cannot be dropped or renamed.
 DROP TABLE t1;
 create table t1 (i int, j double DEFAULT (i * i) ) ;
 # "Unknown column 'z' in 'default value expression'"

mysql-test/r/functional_index.result

@@ -684,9 +684,9 @@ ERROR HY000: Cannot create a functional index on an expression that returns a BL
 # column that is a part of a functional index.
 CREATE TABLE t1 (col1 INT, col2 INT, col3 INT, INDEX idx ((col1 + col2 + col3)));
 ALTER TABLE t1 DROP COLUMN col2;
-ERROR HY000: Cannot drop column 'col2' because it is used by a functional index. In order to drop the column, you must remove the functional index.
+ERROR HY000: Column 'col2' has a functional index dependency and cannot be dropped or renamed.
 ALTER TABLE t1 DROP COLUMN col3;
-ERROR HY000: Cannot drop column 'col3' because it is used by a functional index. In order to drop the column, you must remove the functional index.
+ERROR HY000: Column 'col3' has a functional index dependency and cannot be dropped or renamed.
 ALTER TABLE t1 DROP INDEX idx;
 ALTER TABLE t1 DROP COLUMN col3;
 DROP TABLE t1;

mysql-test/suite/gcol/r/gcol_bugfixes.result

@@ -2765,9 +2765,9 @@ SHOW CREATE TABLE t_gen;
 ALTER TABLE t_gen RENAME COLUMN a TO c, CHANGE COLUMN b b DOUBLE GENERATED ALWAYS AS (SQRT(c));
 SELECT * FROM t_gen;
 SHOW CREATE TABLE t_gen;
---error ER_BAD_FIELD_ERROR
+--error ER_DEPENDENT_BY_GENERATED_COLUMN
 ALTER TABLE t_gen CHANGE COLUMN c a INT;
---error ER_BAD_FIELD_ERROR
+--error ER_DEPENDENT_BY_GENERATED_COLUMN
 ALTER TABLE t_gen RENAME COLUMN c TO a;
 DROP TABLE t_gen;
 

mysql-test/suite/gcol/t/gcol_bugfixes.test

@@ -839,14 +839,12 @@ DROP TABLE t2;
 --echo # BUG#28040739 - WL#9418: INCORRECT ERROR THROWN WITH RENAME COLUMN
 --echo #
 CREATE TABLE t1 ( i INT, j INT DEFAULT( i * i) ) ;
---echo # "Unknown column 'i' in 'default value expression'"
---error 1054
+--error ER_DEPENDENT_BY_DEFAULT_GENERATED_VALUE
 ALTER TABLE t1 RENAME COLUMN i to i1 ;
 DROP TABLE t1;
 
 CREATE TABLE t1 ( i INT, j INT DEFAULT (i * i) ) ;
---echo # "Unknown column 'i' in 'default value expression'"
---error 1054
+--error ER_DEPENDENT_BY_DEFAULT_GENERATED_VALUE
 ALTER TABLE t1 CHANGE COLUMN i i1 DOUBLE DEFAULT ( 4 * 4 ) ;
 DROP TABLE t1;
 
@@ -898,4 +896,4 @@ CREATE TABLE t1 (a datetime, b varchar(10) DEFAULT (localtimestamp()));
 --error ER_DATA_TOO_LONG
 INSERT INTO t1(a) VALUES (now());
 SELECT * FROM t1;
-DROP TABLE t1;
+DROP TABLE t1;

mysql-test/t/alter_table.test

@@ -395,9 +395,9 @@ CREATE TABLE t1 (
 --echo # See that we get a reasonable error message when trying to remove a
 --echo # column that is a part of a functional index.
 CREATE TABLE t1 (col1 INT, col2 INT, col3 INT, INDEX idx ((col1 + col2 + col3)));
---error ER_CANNOT_DROP_COLUMN_FUNCTIONAL_INDEX
+--error ER_DEPENDENT_BY_FUNCTIONAL_INDEX
 ALTER TABLE t1 DROP COLUMN col2;
---error ER_CANNOT_DROP_COLUMN_FUNCTIONAL_INDEX
+--error ER_DEPENDENT_BY_FUNCTIONAL_INDEX
 ALTER TABLE t1 DROP COLUMN col3;
 ALTER TABLE t1 DROP INDEX idx;
 ALTER TABLE t1 DROP COLUMN col3;

mysql-test/t/default_as_expr.test

@@ -8629,7 +8629,7 @@ ER_GENERATED_COLUMN_VARIABLES
   eng "Expression of generated column '%s' cannot refer user or system variables."
 
 ER_DEPENDENT_BY_DEFAULT_GENERATED_VALUE
-  eng "Column '%s' of table '%s' has a default value expression dependency and cannot be dropped."
+  eng "Column '%s' of table '%s' has a default value expression dependency and cannot be dropped or renamed."
 
 ER_DEFAULT_VAL_GENERATED_NON_PRIOR
   eng "Default value expression of column '%s' cannot refer to a column defined after it if that column is a generated column or has an expression as default value."
@@ -8844,6 +8844,9 @@ ER_UNSUPPORTED_USE_OF_GRANT_AS
 ER_UKNOWN_AUTH_ID_OR_ACCESS_DENIED_FOR_GRANT_AS
   eng "Either some of the authorization IDs in the AS clause are invalid or the current user lacks privileges to execute the statement."
 
+ER_DEPENDENT_BY_FUNCTIONAL_INDEX
+  eng "Column '%s' has a functional index dependency and cannot be dropped or renamed."
+
 #
 #  End of 8.0 error messages.
 #

mysql-test/t/functional_index.test

@@ -2958,6 +2958,12 @@ class Alter_inplace_info {
   KEY_PAIR *index_rename_buffer;
   KEY_PAIR *index_altered_visibility_buffer;
 
+  /** Number of virtual columns to be added. */
+  uint virtual_column_add_count;
+
+  /** number of virtual columns to be dropped. */
+  uint virtual_column_drop_count;
+
   /**
      Context information to allow handlers to keep context between in-place
      alter API calls.
@@ -3033,6 +3039,8 @@ class Alter_inplace_info {
         index_rename_count(0),
         index_altered_visibility_count(0),
         index_rename_buffer(NULL),
+        virtual_column_add_count(0),
+        virtual_column_drop_count(0),
         handler_ctx(NULL),
         group_commit_ctx(NULL),
         handler_flags(0),

share/errmsg-utf8.txt

@@ -365,10 +365,7 @@ class Alter_info {
   };
 
   /**
-     Columns and keys to be dropped.
-     After mysql_prepare_alter_table() it contains only foreign keys and
-     virtual generated columns to be dropped. This information is necessary
-     for the storage engine to do in-place alter.
+     Columns, keys and constraints to be dropped.
   */
   Mem_root_array<const Alter_drop *> drop_list;
   // Columns for ALTER_COLUMN_CHANGE_DEFAULT.