Skip to content

Commit

Permalink
MDEV-16329 [5/5] ALTER ONLINE TABLE
Browse files Browse the repository at this point in the history
* Log rows in online_alter_binlog.
* Table online data is replicated within dedicated binlog file
* Cached data is written on commit.
* Versioning is fully supported.
* Works both wit and without binlog enabled.

* For now savepoints setup is forbidden while ONLINE ALTER goes on.
  Extra support is required. We can simply log the SAVEPOINT query events
  and replicate them together with row events. But it's not implemented
  for now.

* Cache flipping:

  We want to care for the possible bottleneck in the online alter binlog
  reading/writing in advance.

  IO_CACHE does not provide anything better that sequential access,
  besides, only a single write is mutex-protected, which is not suitable,
  since we should write a transaction atomically.

  To solve this, a special layer on top Event_log is implemented.
  There are two IO_CACHE files underneath: one for reading, and one for
  writing.

  Once the read cache is empty, an exclusive lock is acquired (we can wait
  for a currently active transaction finish writing), and flip() is emitted,
  i.e. the write cache is reopened for read, and the read cache is emptied,
  and reopened for writing.

  This reminds a buffer flip that happens in accelerated graphics
  (DirectX/OpenGL/etc).

  Cache_flip_event_log is considered non-blocking for a single reader and a
  single writer in this sense, with the only lock held by reader during flip.

  An alternative approach by implementing a fair concurrent circular buffer
  is described in MDEV-24676.

* Cache managers:
  We have two cache sinks: statement and transactional.
  It is important that the changes are first cached per-statement and
  per-transaction.
  If a statement fails, then only statement data is rolled back. The
  transaction moves along, however.

  Turns out, there's no guarantee that TABLE well persist in
  thd->open_tables to the transaction commit moment.
  If an error occurs, tables from statement are purged.
  Therefore, we can't store te caches in TABLE. Ideally, it should be
  handlerton, but we cut the corner and store it in THD in a list.
  • Loading branch information
FooBarrior authored and vuvova committed Jun 18, 2022
1 parent a843443 commit 06ee4bb
Show file tree
Hide file tree
Showing 42 changed files with 1,837 additions and 181 deletions.
2 changes: 2 additions & 0 deletions mysql-test/include/have_log_bin_off.require
@@ -0,0 +1,2 @@
Variable_name Value
log_bin OFF
2 changes: 0 additions & 2 deletions mysql-test/main/alter_table.result
Expand Up @@ -1756,7 +1756,6 @@ info: Records: 0 Duplicates: 0 Warnings: 1
Warnings:
Note 1831 Duplicate index `i3`. This is deprecated and will be disallowed in a future release
ALTER TABLE t1 ADD INDEX i4(b), ALGORITHM= COPY, LOCK= NONE;
ERROR 0A000: LOCK=NONE is not supported. Reason: COPY algorithm requires a lock. Try LOCK=SHARED
ALTER TABLE t1 ADD INDEX i5(b), ALGORITHM= COPY, LOCK= SHARED;
affected rows: 2
info: Records: 2 Duplicates: 0 Warnings: 1
Expand All @@ -1774,7 +1773,6 @@ ERROR 0A000: LOCK=NONE/SHARED is not supported for this operation. Try LOCK=EXCL
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= INPLACE, LOCK= EXCLUSIVE;
affected rows: 0
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= COPY, LOCK= NONE;
ERROR 0A000: LOCK=NONE is not supported. Reason: COPY algorithm requires a lock. Try LOCK=SHARED
ALTER ONLINE TABLE m1 ADD COLUMN c int;
ERROR 0A000: LOCK=NONE is not supported for this operation. Try LOCK=SHARED
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= COPY, LOCK= SHARED;
Expand Down
10 changes: 8 additions & 2 deletions mysql-test/main/alter_table.test
Expand Up @@ -1528,8 +1528,12 @@ ALTER TABLE t1 DROP INDEX i1, DROP INDEX i2, DROP INDEX i3, DROP INDEX i4;
ALTER TABLE t1 ADD INDEX i1(b), ALGORITHM= INPLACE, LOCK= NONE;
ALTER TABLE t1 ADD INDEX i2(b), ALGORITHM= INPLACE, LOCK= SHARED;
ALTER TABLE t1 ADD INDEX i3(b), ALGORITHM= INPLACE, LOCK= EXCLUSIVE;
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
--disable_info
--disable_warnings
--error 0,ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
ALTER TABLE t1 ADD INDEX i4(b), ALGORITHM= COPY, LOCK= NONE;
--enable_warnings
--enable_info
ALTER TABLE t1 ADD INDEX i5(b), ALGORITHM= COPY, LOCK= SHARED;
ALTER TABLE t1 ADD INDEX i6(b), ALGORITHM= COPY, LOCK= EXCLUSIVE;

Expand All @@ -1538,8 +1542,10 @@ ALTER TABLE m1 ENABLE KEYS, ALGORITHM= INPLACE, LOCK= NONE;
--error ER_ALTER_OPERATION_NOT_SUPPORTED
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= INPLACE, LOCK= SHARED;
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= INPLACE, LOCK= EXCLUSIVE;
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
--disable_info
--error 0,ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
ALTER TABLE m1 ENABLE KEYS, ALGORITHM= COPY, LOCK= NONE;
--enable_info
--error ER_ALTER_OPERATION_NOT_SUPPORTED
ALTER ONLINE TABLE m1 ADD COLUMN c int;
# This works because the lock will be SNW for the copy phase.
Expand Down
1 change: 0 additions & 1 deletion mysql-test/main/alter_table_locknone.result
Expand Up @@ -71,7 +71,6 @@ ERROR 0A000: LOCK=NONE is not supported for this operation. Try LOCK=SHARED
alter online table t1 add f int;
ERROR 0A000: LOCK=NONE is not supported for this operation. Try LOCK=SHARED
alter online table t1 engine=memory;
ERROR 0A000: LOCK=NONE is not supported. Reason: COPY algorithm requires a lock. Try LOCK=SHARED
alter online table t1 rename to t2;
ERROR 0A000: LOCK=NONE/SHARED is not supported for this operation. Try LOCK=EXCLUSIVE
alter online table t1 checksum=1;
Expand Down
2 changes: 1 addition & 1 deletion mysql-test/main/alter_table_locknone.test
Expand Up @@ -77,7 +77,7 @@ alter online table t1 modify c varchar(50);
alter online table t1 modify c varchar(100);
--error ER_ALTER_OPERATION_NOT_SUPPORTED
alter online table t1 add f int;
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
--error 0,ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
alter online table t1 engine=memory;
--error ER_ALTER_OPERATION_NOT_SUPPORTED
alter online table t1 rename to t2;
Expand Down
2 changes: 2 additions & 0 deletions mysql-test/main/alter_table_online.combinations
@@ -0,0 +1,2 @@
[innodb]
[rocksdb]

0 comments on commit 06ee4bb

Please sign in to comment.